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ABSTRACT 


Annual  appraisals  across  corporate  America  range  from  going  through  the  motions  of 
filling  out  paperwork  to  an  arduous  task  of  force  ranking  every  employee  against  each  other. 

The  primary  goal  is  to  use  a  perfonnance  system  that  can  fairly  evaluate  performance  that  is  both 
helpful  to  the  organization  and  equitable  for  the  individual  employee.  The  Department  of 
Defense  (DoD)  has  attempted  to  improve  human  resource  rewards  and  recognition  with  annual 
appraisals  using  several  pay-for-performance  models,  including  the  National  Security  Personnel 
System  (NSPS).  Whether  elements  of  this  or  any  other  pay-for-perfonnance  system  can  provide 
a  fair  mechanism  of  evaluation  and  incentivize  the  workforce  will  remain  in  debate  for  some 
time. 

This  paper  will  investigate  the  consistency  of  evaluations  given  to  over  1,500  employees 
within  a  DoD  command  geographically  dispersed  in  eight  locations  across  the  country.  Each  of 
the  eight  separate  locations  has  its  own  specialized  areas,  but  all  work  in  a  particular  technical 
field  for  the  defense  industry.  NSPS  provides  a  standard  way  to  evaluate  the  federal  workforce, 
but  also  allows  greater  organization  flexibility  on  many  human  resource  tasks  like  recruitment, 
pay  setting,  promotions,  and  details  on  annual  perfonnance  appraisals.  The  headquarters 
element  of  the  observed  command  evaluates  the  flexible  NSPS  business  rules  for  each  of  their 
subordinate  commands  to  ensure  the  right  level  of  standardization  exists.  Therefore,  the  expected 
goal  would  be  that  a  level  of  consistency  should  exist  between  the  eight  subordinate  commands. 
Using  evaluation  outcomes  for  every  employee  from  the  previous  three  years  since  this 
command  began  transition  to  NSPS,  this  paper  will  compare  the  resultant  performance  ratings 
and  resultant  salary  increases  and  one-time  bonus  awards.  Correlations  between  performance 
scores  between  seniority  pay  bands  (intern,  full  performance,  and  senior  level)  as  well  has 
hierarchical  (student  interns,  support  staff,  professional,  and  managerial)  will  also  be  investigated 
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for  variability.  Although  representing  a  very  small  sliver  of  the  DoD  workforce,  these  results 
may  indicate  some  of  the  positive  aspects  of  NSPS  as  a  pay-for-perfonnance  system  as  well  as 
some  areas  in  need  of  improvement. 


CHAPTER  1 
INTRODUCTION 


Introduction  and  Preliminary  Study 

The  National  Security  Personnel  System  (NSPS)  was  designed  to  successfully 
incorporate  a  pay-for-perfonnance  annual  evaluation  for  each  Department  of  Defense  (DoD) 
employee.  The  goal  was  to  create  a  process  that  would  directly  link  successful  performance  of 
an  employee  to  monetary  salary  increases  and  bonuses.  This  also  would  be  a  strategic  human 
resource  practice  that  could  make  DoD  provide  more  competitive  pay  when  compared  with 
industry.  It  has  several  standard  procedures  but  also  allows  each  organization  the  flexibility  to 
tailor  the  system  to  their  performance  and  human  resource  needs.  Through  the  use  of  business 
rules,  that  flexibility  can  be  outlined  and  provide  a  customized  structure  for  how  NSPS  will 
specifically  be  used  for  any  organization.  Those  business  rules  provide  the  final  structure  that 
allow  each  employee  and  supervisor  to  have  a  common  language  and  known  procedure  on  how 
annual  perfonnance  evaluations  and  resulting  payout  of  salary  increase  and  bonus  will  occur. 

When  a  large  organization  is  dispersed  across  multiple  locations,  each  with  their 
specialized  ways  of  doing  business,  it  may  provide  a  greater  challenge  to  balance  consistent 
standards  with  customized  needs.  This  paper  will  look  at  one  such  dispersed  organization  and 
investigate  the  similarity  of  past  employee  evaluations  to  check  for  overall  consistency  within 
those  reviews  and  identify  possible  areas  of  improvement.  The  organization,  selected  for  this 
study  perfonns  many  different  types  of  technical  services  for  the  U.S.  Government.  In  general, 
each  location  organically  evaluates  the  performance  of  all  of  their  government  employees  using 
the  same  rating  scale  and  similar  objectives.  Some  locations  had  so  many  employees,  that  they 
had  to  break  employees  into  separate  review  groups.  These  separate  review  groups  were 
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combined  as  a  single  location  for  the  purpose  of  this  study.  Total  data  for  employee  reviews  was 
collected  from  nine  separate  locations  and  analyzed  for  consistency. 

Research  Hypothesis 

Even  though  all  of  the  subordinate  commands  perfonn  similar  functions  for  DoD,  each 
has  its  own  specialized  mission  area.  This  coupled  with  the  geographic  dispersion  across  the 
United  States  promotes  each  subordinate  command  developing  individualized  best  business 
practices  and  organization  culture.  Therefore,  the  hypothesis  is  that  the  pay-for-performance 
summarized  results  will  reveal  inconsistency  and  indicate  room  for  improvement  in  providing  a 
more  standardized  approach  across  the  entire  organization. 

Research  Methods 

Performance  review  data  on  every  government  employee  evaluated  was  collected  from  9 

different  locations  for  two  years.  The  resulting  dataset  included  over  2,500  data  points;  a  sample 

of  that  data  is  illustrated  in  Table  1.  Each  line  of  data  represents  a  single  employee’s 

performance  review,  including  which  location  they  work  at,  when  the  review  was  conducted,  the 

employee’s  job  position,  and  their  resulting  performance  rating  infonnation.  At  each  location, 

all  employees  are  evaluated  in  pay  pools,  designated  as  “PP”  in  Table  1  from  1-9.  During  a  pay 

pool,  a  group  of  higher-level  managers  get  together  and  review  every  employee  within  their  pool, 

typically  using  a  standard  form  that  states  an  employee’s  annual  objectives  and  both  the  direct 

supervisor’s  and  the  employee’s  input  on  their  perfonnance  for  that  year.  In  this  research,  a  total 

number  of  nine  different  locations  or  pay  pools  were  compared.  Two  years  of  data  were 

included  in  this  research.  A  given  annual  performance  cycle  begins  in  October  and  ends  in  the 

September  of  the  following  year;  the  next  three  months  allow  time  for  writing  the  reviews  and 

evaluating  all  employees  in  a  pay  pool.  The  results  are  then  provided  to  the  employee  as  a 

scored  evaluation,  a  salary  increase,  and  a  onetime  annual  bonus  provided  by  January  of  the  next 
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year.  Within  the  data,  as  shown  in  Table  1,  the  year  is  designated  in  the  “YR”  column;  “09” 
means  the  employee  was  given  his/her  payout  of  salary  and/or  bonus  in  January  of  2009 
resulting  from  the  pay  pool’s  assessment  on  his/her  performance  from  October  2007  to 
September  2008. 

Table  1:  Excerpt  of  NSPS  Raw  Data 
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1 

45.5 

1 

09 

YE 

S 
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3 

1 

60 

1 

09 

YE 

S 

2 

3 

2 

22.7 

Additional  independent  variables  include  information  about  the  employee.  Every 
employee  is  classified,  through  their  job  position  description,  by  a  pay  schedule  (PS)  and  pay 
band  (PB).  The  work  group  (WG)  is  an  abbreviation  of  an  employee’s  pay  schedule  grouping 
those  with  similar  positions,  including  students,  staff,  and  supervisor  employees.  Table  2 
describes  how  the  pay  schedule  designation  indicates  the  type  of  work  any  particular  employee 
performs. 
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Table  2:  Direct  Relationship  Between  Work  Group  and  Pay  Schedule 


Work  Group  (WG),  4lh  column 

Pay  Schedule  (PS),  3rd  column 

Interns  (1) . . . 

include  all  full  time  students  of  any  field,  designated  by  YP.  These  are 

typically  summer  students  and  co-op  students  that  may  work  full  time 

for  a  few  months  a  year. 

Support  Staff  (S)  ... 

includes  administrative  personnel,  designated  by  YB,  as  well  as 

technical  staff,  designated  by  YE. 

Professional  staff  (P)  ... 

includes  analysts,  designated  by  YA,  as  well  as  scientists  and 

engineers,  designated  by  YD. 

Managers  (M)  ... 

include  any  supervisor  combining  analysts  and  administrative 

supervisors  (Y C)  as  well  as  scientist,  engineering,  and  technical 

supervisors  (YF). 

Pay  band  (PB)  in  Table  1  designates  the  seniority  or  skill  level  of  an  employee  within  a 
given  position.  Pay  band  1  employees  are  students  and  interns  usually  spending  a  few  years 
learning  the  details  of  the  job  they  are  in.  At  Pay  band  2,  the  employee  is  at  “full  performance” 
and  is  considered  experienced  at  the  position  they  are  holding.  Pay  band  3  indicates  a  senior- 
level  employee,  highly  skilled  with  multiple  years  of  experience.  For  the  technical  staff,  YE  pay 
schedule,  there  exists  an  additional  pay  band  4  indicating  a  master  skilled  employee  within  that 
technical  field. 

Dependent  Variables 

Upon  completion  of  a  given  year’s  pay  pool,  employees  are  provided  verbal  and  written 
feedback  on  the  details  of  their  performance.  They  also  receive  the  same  three  quantitative 
values  summarizing  their  perfonnance  for  that  pay  pool. 
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Rating  (RT)  -  As  a  result  of  individually  rating  each  objective  for  an  employee,  a 
composite  rating  is  calculated.  This  provides  an  overall  level  of  performance  accomplished  for 
that  rating  period:  1  -  Unsatisfactory,  2  -  Fair,  3  -  Valued  Employee,  4  -  Exceed  Expectations, 
and  5  -  Role  Model.  The  higher  the  final  rating,  the  more  esteemed  that  employee’s 
perfonnance  was  considered  during  the  rating  cycle. 

Shares  Awarded  (SH)  -  Directly  correlated  to  the  rating  is  the  number  of  shares  provided 
to  the  employee.  This  will  translate  into  how  much  money  will  be  provided  as  a  total  between 
salary  increase  or  bonus  for  that  year.  An  employee  ranked  as  a  role  model  can  receive  5  or  6 
shares,  exceed  expectations  receives  3  or  4  shares,  a  valued  employee  receives  1  or  2  shares,  and 
no  share  or  perfonnance  payout  is  provided  to  an  unsatisfactory  or  fair  employee.  The  more 
shares  a  particular  employee  is  assigned,  the  greater  the  financial  payout  for  that  year.  However, 
the  final  payout  takes  into  account  all  other  employees’  shares  within  the  pay  pool.  If  all 
employees  within  a  pay  pool  were  assigned  a  high  number  of  shares,  their  individual  payout 
would  be  significantly  less. 

Salary  Split  Percentage  (S%)  -  When  a  monetary  value  for  the  perfonnance  payout  is 
calculated  using  the  number  of  shares  for  an  employee,  the  percentage  split  indicates  how  much 
of  that  total  payout  becomes  an  increase  in  salary;  the  balance  of  the  payout  will  be  paid  as  a 
onetime  bonus  for  that  year.  Example,  if  a  single  share  for  an  employee  is  worth  $500  and  that 
employee  receive  two  shares,  his/her  total  payout  would  be  (2x$500)  $1,000.  If  an  employee  has 
a  40  percent  salary  split,  $400  dollars  would  be  added  to  the  employee’s  base  salary  and  the 
remaining  $600  would  be  a  bonus  issued  on  the  first  check  of  the  new  year. 

Each  of  the  nine  pay  pools  use  the  same  overall  organizational  business  rules  for 
administering  NSPS  ratings,  shares,  and  salary  splits.  These  three  resulting  variables  were  used 
to  gage  the  level  of  consistency  implemented  by  this  nationwide  organization. 
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RESULTS  -  Positive  Indications  of  Consistency 

The  Civilian  Personnel  Management  System  (2009)  published  overall  NSPS  average 
ratings  for  over  1 10,000  employees.  The  ratings  were  recorded  in  January  2008  for  the  work 
perfonned  the  previous  year,  and  shown  in  Figure  1  as  “2008  Global.”  The  combined 
organization’s  overall  averages  with  748  rated  employees  in  2008  and  1,680  rated  employees  in 
2009  fall  along  the  same  distribution  as  the  global  average.  The  2009  results  more  closely  match 
the  2008  global  average  with  slightly  more  “exceed  expectations”  ratings  offset  by  lower  “valued 
employee”  ratings.  This  result  indicates  that  the  command  as  a  whole  has  become  more 
consistent  compared  to  the  many  other  government  organizations  using  NSPS. 

Figure  1:  Command  NSPS  Ratings  Distribution  Compared  to  Global  Distribution  in  2008 


Figure  2  shows  differences  across  the  command  for  the  different  pay  bands.  All  of  the 

nine  pay  pools  were  combined,  and  these  two  graphs  show  the  difference  in  average  rating  and 

salary  split  percentages  across  the  experience  level  of  the  workforce.  In  a  perfectly  consistent 

scenario,  all  of  the  values  on  the  graph  would  be  zero,  indicating  that  the  average  rating  and 

share  percentages  were  exactly  the  same  regardless  of  which  pay  band  an  employee  was  in. 

NSPS  was  designed  so  that  employees’  performance  would  take  expected  skill  level  into 

account.  Therefore,  if  two  employees  had  similar  perfonnance,  the  pay  band  1  employee  would 

6 


likely  be  exceeding  expectations  since  he/she  is  working  at  a  higher  performance  level. 

Likewise,  a  pay  band  2  employee  working  at  a  similar  overall  performance  level  may  be  coached 
to  improve.  In  2008,  the  most  master  technicians  (pay  band  4)  seem  to  have  a  much  higher 
rating  than  the  overall  workforce,  but  in  2009,  all  pay  bands  seem  to  be  normalized.  The  rating 
data,  in  Figure  2  (left),  flattening  during  2009  could  indicate  that  comparisons  between 
employees,  even  those  with  different  skill  levels,  are  better  following  the  intended  standards. 
Figure  2:  Average  Difference  Between  Command  Average  and  Pay  Band  for  NSPS  Rating 
(Left)  and  Share  Percentage  Toward  Salary  (Right) 


RESULTS  -  Mixed  Indications  of  Consistency 

In  contrast,  the  right  graph  of  Figure  2  indicates  a  possible  wider  variance  across  pay 
bands  when  looking  at  the  percentage  of  an  employee’s  payout  that  will  increase  his/her  annual 
salary.  It  was  expected  that  pay  band  1  employee  would  have  a  significantly  lower  share  split 
percentage.  Pay  band  1  includes  full-time  students  who  work  part  time  and  interns  who  have 
little  previous  experience.  NSPS  states  that  these  pay  band  1  employees  should  get  all  of  their 
payout  in  a  one-time  bonus.  Salary  increases  for  these  employees  does  not  come  from  NSPS  but 
other  incentive  mechanisms.  Therefore,  pay  band  1  employees  would  be  expected  to  have  30-40 
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percent  lower  splits  than  the  overall  workforce  average.  However,  the  difference  between  pay 
band  2  (full  perfonnance),  3  (senior  level),  and  4  (master  technicians)  seems  to  have  increased, 
although  not  significantly  from  the  previous  year.  Additional  data  of  subsequent  years  may  help 
detennine  if  this  trend  is  becoming  more  consistent  or  if  a  management  intervention  should  be 
considered. 

Figure  1  shows  the  average  rating  for  the  entire  organization  increased  from  3.36  in  2008 
to  3.41  in  2009.  Only  three  locations  implemented  NSPS  in  2008  while  all  eight  locations 
completed  pay-for-performance  evaluations  in  2009.  Having  such  a  small  change  in  average 
rating  may  actually  indicate  a  fairly  consistent  first  implementation  for  the  five  locations  in  2009. 
Figure  3  (left)  shows  the  eight  geographically  dispersed  locations’  deviations  from  that  average 
range  from  3.33  for  pay  pool  2  (PP2)  to  3.72  from  pay  pool  6  (PP6).  This  resulted  in  a  range  in 
average  share  from  2.24  for  PP2  to  3.1 1  PP6.  The  total  difference  of  0.87  shares  between  PP2 
and  PP6  could  result  in  an  additional  payout  of  $700  or  more.  Data  outlining  the  payout  amounts 
for  each  of  the  pay  pools  was  not  provided;  therefore,  any  financial  benefit  from  having  a  higher 
rating  or  shares  could  not  be  verified.  Reviewing  Figure  3  (left),  pay  pools  3  and  6  may  be 
outliers  but  the  rest  of  the  subordinate  commands  seem  to  fall  within  the  overall  average.  Six  out 
of  the  eight  locations  seem  be  relatively  consistent  with  each  other  on  assigning  ratings  and 
shares.  Additional  financial  data  and  perfonning  a  trend  analysis  in  subsequent  years  will  better 
indicate  consistency. 
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Figure  3:  Differences  Between  Each  Pay  Pool  and  the  Overall  Command's  Average  in 


NSPS  Rating  and  Share  (Left)  and  Share  Percentage  Toward  Salary  (Right) 
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The  difference  in  share  percentage  (S%),  shown  in  Figure  3  (right),  can  become  a 
balancing  factor  since  it  indicates  the  amount  of  the  payout  that  will  be  placed  into  salary,  with 
the  balance  going  as  a  onetime  bonus.  Although  pay  pool  2  (PP2)  has  the  lowest  rating  and 
share  values,  it  has  the  highest  average  share  percentage:  76.3  percent.  Therefore,  although 
employees  of  pay  pool  2  (PP2)  receive  fewer  shares,  they  may  have  larger  payouts  and  the 
majority  of  that  payout  went  to  improving  their  base  salary.  Pay  pool  3  (PP3),  on  the  other  hand, 
had  the  lowest  average  share  percentage  (37.6  percent),  so  although  more  shares  were  provided, 
the  majority  of  the  payout  went  to  a  onetime  bonus.  Without  the  resulting  information  about 
each  employee’s  payout  in  salary  and  bonus,  only  conjecture  can  be  made  on  the  true 
consistency  of  the  differences  in  pay  pool  results  across  the  eight  locations.  However, 
attempting  to  have  more  consistent  values  of  rating,  shares,  and  salary  percentages  could  be 
achieved  with  more  standardization  over  time,  which  would  likely  lead  to  more  consistent  and 
understood  salary  and  bonus  numbers  for  employees. 
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Aggregating  across  all  the  pay  pools,  variations  were  found  in  reference  to  different  job 
positions.  Lower  average  ratings  and  shares  were  exhibited  for  the  YD  (scientist  and  engineers), 
YE  (technicians),  and  YP  (student)  employees,  illustrated  in  Figure  4.  It  may  be  expected  that 
the  student  employees,  YP,  would  have  lower  ratings  and  shares  since  the  primary  number  of 
them  only  work  for  part  of  the  year  and  may  therefore  have  less  time  to  make  significant 
contributions  toward  the  command’s  goals.  The  share  percentage  for  YP  students  was  not 
included  since  that  average  was  49  percent  below  the  command  average  of  59  percent;  this  is  due 
to  the  NSPS  guideline  that  students,  in  general,  do  not  get  salary  increases  from  NSPS  annual 
payouts  but  by  another  mechanism.  Share  percentage  seems  to  slightly  favor  the  supervisor  pay 
schedules,  YC  and  YF.  This  may  be  due  to  their  position  being  closer  to  top  management  overall 
goals,  therefore  greater  opportunity  exists  to  demonstrate  perfonnance  toward  objectives.  These 
differences  are  relatively  small  and  may  normalize  when  additional  years  of  evaluation  occur. 
Summary  of  Positive  and  Mixed  Findings 

1 .  Between  2008  and  2009,  overall  organizational  average  employee  perfonnance  ratings, 
number  of  shares,  and  salary  percentages  seem  to  be  converging  toward  better  consistency. 

2.  Six  out  of  the  eight  locations  have  similar  values  of  average  rating,  number  of  shares,  and 
salary  percentages. 

3.  Across  all  eight  locations,  supervisors  (YF  and  YC)  have  higher-than-average  ratings, 
number  of  shares,  and  salary  percentages.  In  comparison,  scientists  and  engineers  (YD), 
technicians  (YE)  and  students  (YP)  have  lower  perfonnance  scores.  However,  these 
differences  are  relatively  small  and  may  only  be  noise  within  the  data. 

Although  this  initial  data  may  point  to  a  few  findings,  additional  information  is  necessary 
to  determine  if  consistency  across  the  different  eight  locations  needs  to  be  addressed. 

Subsequent  data  for  following  years  will  either  confinn  that  trends  are  converging  or  will 
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indicate  areas  for  improvement.  More  important,  financial  infonnation  providing  the  amount  of 
salary  and  bonus  dollars  awarded  for  employees  with  varying  ratings,  number  of  shares,  and 
salary  percentages  would  indicate  the  true  metric  for  consistency.  In  most  cases  employees 
desire  two  things  from  a  positive  annual  review:  a  high  rating  score  indicating  his/her  efforts 
exceeded  expectations  and  the  accompany  salary  increase.  With  both  of  those  pieces  of 
information,  a  much  stronger  analysis  could  be  made. 

Figure  4:  Difference  Between  the  Command  for  Each  Pay  Schedule  Employee  Type  for 
NSPS  Rating  and  Share  (Left)  and  Share  Percentage  Toward  Salary  (Right) 
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CHAPTER  2 
PROPOSED  PLAN 


Performance  appraisals  can  be  used  for  two  competing  aspects:  one  can  focus  on  how 
best  to  develop  an  employee  and  another  is  to  assist  in  administration  tasks  like  detennining  pay 
raises,  promotion  opportunities,  and  disciplinary  actions  (Mathis  &  Jackson,  2008,  pg.  333).  As  a 
pay-for-perfonnance  appraisal,  NSPS  is  used  to  recognize  an  employee’s  success  and  apply  an 
increase  to  his/her  salary  based  on  that  success.  Like  other  appraisal  systems,  NSPS  can  also  be 
used  to  determine  the  possibility  of  advancement  and  identify  opportunities  to  grow  the 
employee. 

Herberg’s  Motivation-Hygiene  Factors 

Herzberg’s  Motivation-Hygiene  Theory  separates  factors  into  two  distinct  categories  that 
can  affect  employee  morale.  One  set  of  factors  can  promote  job  satisfaction  and  includes 
recognition  for  achievement,  advancement,  and  the  possibility  of  occupational  growth  (Herzberg, 
1965,  pg.  395).  Completely  different  factors  are  more  responsible  for  job  dissatisfaction 
including  how  employees  are  supervised,  working  conditions,  and  salary  (Herzberg,  1965,  pg. 
395).  As  an  example  illustrating  the  subtle  difference,  using  a  satisfier  factor  of  providing  an 
employee  solid  recognition  for  his/her  performance  will  likely  increase  an  employee’s 
satisfaction;  on  the  other  hand,  less  recognition  may  not  affect  the  employee’s  motivation. 
Conversely,  improving  a  dissatisfier  factor  like  providing  a  large  salary  may  not  change  an 
employee’s  morale;  but  if  the  employee  feels  his/her  salary  is  low,  it  will  likely  drive  his/her 
motivation  down  quickly. 

What  most  employees  desire  from  NSPS  is  first  the  highest  salary  increase  possible 
(salary  dissatisfier),  second  the  highest  performance  rating  reasonable  (recognition  satisfier), 
third  the  highest  bonus  possible  (salary  dissatisfier),  and  finally  possibilities  for  advancement 
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and  growth  (satisfiers).  Therefore  NSPS,  like  many  other  pay-for-performance  systems,  relates 
to  both  factors,  but  first,  it  is  likely  to  be  a  dissatisfier  with  unappreciated  salary  increases  than  as 
a  satisfier  of  appropriate  recognition. 

Proposed  Plan  -  Develop  an  Annual  Monitoring  Process  to  Review  (AMP-Rev) 
Performance  Results 

Since  NSPS  can  have  such  a  powerful  effect  on  employee  morale  that  drives  both 
dissatisfaction  as  well  as  satisfaction  a  tool,  AMP-Rev,  should  be  created  to  monitor  NSPS 
results  to  safeguard  from  potential  dissatisfaction.  This  will  allow  the  command  headquarters  to 
observe  any  trends  that  may  leave  the  employees  feeling  the  overall  system  is  not  equitable.  The 
monitoring  process  should  be  conducted  each  year,  after  the  annual  perfonnance  reviews  are 
completed.  In  this  way,  the  monitoring  process  would  not  affect  that  year’s  perfonnance 
reporting  results  but  could  be  used  to  create  any  needed  corrective  guidance  to  all  the  pay  pool 
locations.  That  guidance  must  then  be  communicated  to  all  employees  in  a  timely  manner  so 
they  can  understand  and  adapt  to  any  changes  made  to  the  pay-for-performance  system  for  the 
next  perfonnance  cycle. 

This  monitoring  process  will  be  similar  to  the  analysis  performed  in  the  Prior  Research 
section.  “The  most  informative  approach  to  ratio  analysis  combines  cross-sectional  and  time- 
series  analyses”  (Gitman,  2009,  pg  53).  Cross-sectional  data  analysis  investigates  variations 
across  the  geographically  dispersed  pay  pools  as  well  as  demographic  differences  between 
employees.  Time-series  analysis  evaluates  changes  in  perfonnance  results  over  time.  As 
mentioned,  data  collection  should  be  perfonned  each  year  after  the  annual  review  to  allow  for 
this  time-series  analysis.  Prior  research  was  only  based  on  two  years  of  data.  In  the  most  recent 
year,  2009,  five  of  the  eight  pay  pools  initiated  NSPS;  therefore  no  time-series  analysis  was 
possible  for  the  majority  of  the  locations. 
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Additional  Data  Required  for  Analysis 

During  the  previous  research  in  Part  A,  most  of  the  data  collection  and  analysis  focused 
on  average  employee  performance  ratings,  which  addressed  the  second  major  factor  of  NSPS: 
recognition.  The  primary  factor  of  salary  increase  was  not  able  to  be  addressed,  although  the 
share  percentage  numbers  may  provide  an  indication  that  more  salary  is  being  provided  to  those 
with  higher  percentages.  This  system  would  need  to  include  financial  information  for  each 
employee  so  that  an  analysis  can  make  sure  that  the  salary  increase  distribution  is  equitable 
across  all  pay  pools.  The  following  is  a  list  of  desired  additional  infonnation  that  would  better 
allow  for  combined  cross-sectional  and  time-series  analysis. 

1)  Multiple  pay  pool  data  for  larger  pay  pool  locations  -  Some  of  the  commands  are  so 
large  they  split  their  employees  into  several  smaller  pay  pools.  This  allows  for  a  more 
effective  review  by  lumping  employees  with  similar  job  functions.  Instead  of  just  using 
the  combined,  rolled  up,  results  for  a  single  geographical  location,  it  would  be  useful  to 
check  for  internal  consistency  between  these  localized  pay  pools  at  a  single  geographical 
location. 

2)  Performance  Score  -  NSPS  uses  a  weighted  scoring  system  that  calculates  an 
employee’s  perfonnance  rating.  The  supervisor  and  pay  pool  can  determine  individual 
scores  for  an  employee’s  weighted  objectives,  but  cannot  directly  change  the  resulting 
calculated  performance  rating.  The  ability  to  understand  how  an  employee’s  rating  was 
calculated  may  lead  to  better  internal  consistency  of  the  final  performance  rating  results. 
It  would  be  interesting  to  determine  the  total  number  of  scores  that  were  on  the  border  of 
different  ratings.  Also,  someone  with  a  calculated  score  of  2.5 1-3.50  can  get  either  1  or  2 
shares,  and  shares  are  more  directly  related  to  salary  increases  than  ratings.  This  data 
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will  allow  a  check  to  see  if  the  pay  pools  designate  number  of  shares  differently  when 
considered  against  an  employee’s  calculated  score. 

3)  Performance  Payout  -  NSPS  is  not  about  rating-for-performance  or  shares-for- 
perfonnance;  it  is  about  pay-for-perfonnance.  It  would  be  useful  to  see  how  the  actual 
payout  (combination  of  bonus  and  salary)  plays  out  both  within  a  pay  pool  and  across  the 
command.  Even  more  important  would  be  to  include  the  salary  increase  and  the  one-time 
bonus  that  was  provided  to  each  employee  that  resulted  from  his/her  performance.  Since 
salary  increase  was  designated  as  the  employees’  most  important  result  from  NSPS,  that 
factor  should  be  used  as  a  major  comparison  in  both  cross-sectional  and  time-series 
analysis.  Salary  increase  alone  may  help  monitor  possible  trends  that  could  lead  to 
employee  dissatisfaction. 

4)  Current  Salary  -  To  make  the  payout  dollars  more  useful,  it  would  be  helpful  to  use  the 
employee  current  salary  to  make  additional  calculations.  Current  salary  would  be  used  to 
correlate  with  perfonnance  rating/shares.  Do  higher  paid  employees  typically  get  lower 
ratings  (since  more  is  expected  from  them)?  Do  they  represent  a  majority  of  reduced 
salary  splits  since  they  may  be  coming  close  to  an  established  salary  cap  for  their 
position?  Under  the  previous  annual  appraisal  system,  most  commands  followed  a  very 
structured  schedule  for  providing  fixed  salary  increases.  NSPS  claimed  that  for  high 
perfonners,  they  would  gain  a  larger  salary  increase  when  compared  with  the  old  system. 
Using  currently  salary,  a  comparison  for  each  employee  could  be  made  that  would 
calculate  the  difference  in  salary  increase  between  NSPS  and  the  old  system.  From  the 
prior  research,  it  appeared  that  most  employees  did  enjoy  a  higher  salary  increase  with 
NSPS,  but  this  additional  data  could  confirm  that  finding  and  therefore  be  used  to 
improve  employee  satisfaction  with  the  new  system. 
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5)  Pro-Rate  Salary  Increase  -  For  students  and  employees  who  did  not  work  the  whole 
year,  NSPS  dictates  that  their  perfonnance  payout  should  be  pro-rated.  Therefore,  an 
employee  who  only  worked  half  of  a  year  would  receive  50  percent  of  his/her  calculated 
payout.  This  information  would  ensure  that  their  payout  and  salary  numbers  were  not 
inappropriately  inflated  when  compared  across  time  or  with  other  pay  pool  locations. 
Proposed  AMP-Rev  Plan:  Expected  Outcome  and  Benefits 

Using  a  pay-for-performance  appraisal  system  is  new  for  the  individual  pay  pools  within 
the  command.  With  such  a  large  culture  change,  it  is  expected  that  there  will  be  several 
inconsistencies  identified  when  the  combined  cross-sectional  and  time-series  analysis  is 
performed.  Using  a  tool  like  the  proposed  Annual  Monitoring  Process  to  Review  (AMP-Rev) 
performance  results  should  not  only  be  able  to  illustrate  inconsistencies  but  may  be  used  to 
prioritize  the  most  essential  areas  that  without  intervention  may  lead  employees  to  feel  their 
increases  in  their  salary  are  not  equitable.  Using  this  tool  immediately  after  the  performance 
reviews  have  been  completed  provides  a  mechanism  for  the  most  up-to-date  information  to  be 
used  to  modify  the  overall  command’s  business  rules.  This  will  empower  a  more  proactive 
stance;  identifying  possible  trends  early  will  enable  minor  policy  adjustments  to  take  effect 
before  those  trends  turn  into  a  reality  that  results  in  employee  job  dissatisfaction.  Activating  the 
AMP-Rev  tool  after  a  performance  cycle  will  prevent  any  concern  that  the  analysis  could  be  used 
in  a  retroactive  manner,  which  could  lead  to  employee  distrust  of  management  who  could  be 
influenced  by  early  results. 

Initially,  it  is  expected  that  numerous  inconsistencies  across  the  geographically  dispersed 

pay  pools  will  be  identified.  The  AMP-Rev  tool  can  then  be  used  by  senior  management  to 

detennine  the  most  pressing  issues  and  help  find  the  greatest  leveraging  changes  that  will  lead  to 

more  consistent  results  in  subsequent  years.  Over  time,  both  the  evolving  experience  of  the  pay 
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pools  as  well  as  initiatives  developed  from  AMP-Rev  analysis  should  lead  to  very  consistent  and 
equitable  perfonnance  results  across  the  command.  At  that  time,  the  AMP-Rev  should  remain  in 
place,  primarily  as  a  time-series  analysis  tool,  to  ensure  consistency  is  maintained. 

Proposed  AMP-Rev  Plan:  Constraints 

As  proposed  the  AMP-Rev  tool  is  strictly  a  mathematical  tool  designed  to  investigate 
consistency  across  several  factors  and  over  time.  It  will  identify  variation  across  these 
conditions;  however,  it  will  not  assist  in  providing  valid  explanations  that  may  be  causing  those 
variations.  For  instance  in  the  Prior  Research  section,  the  final  finding  indicated  a  possible 
inconstancy  where  supervisors  are  receiving  both  higher  overall  ratings  and  increase  share 
percentages  toward  salary.  This  variation  could  be  explained  as  a  “good  ole  boy  network,” 
where  the  pay  pool  was  inappropriately  inflating  supervisors’  performance  scores.  These  higher 
scores  may  also  be  explained  due  to  supervisors  having  more  influence  over  parts  of  the 
organization.  They,  therefore,  have  a  greater  ability  to  directly  impact  organization  objectives. 
When  the  organization  is  doing  well,  it  appropriately  reflects  highly  on  those  leaders.  If  the 
organization  was  not  meeting  its  objectives,  it  may  be  expected  that  supervisors  would  have 
much  lower  ratings  since  they  may  not  be  properly  integrating  the  great  work  individual 
employees  were  accomplishing.  These  explanations  for  the  mathematical  variations  would  need 
to  be  detennined  by  an  investigation  and  would  not  involve  AMP-Rev. 

As  with  any  simple  database  tool,  the  best  analysis  does  not  occur  using  the  same 
standardized  reports.  The  most  interesting  findings  will  occur  only  when  an  experienced  analyst 
uses  data  mining  techniques  to  uncover  pertinent  relationships.  As  more  data  is  included  in  the 
tool  as  proposed  above,  the  complexity  of  relationships  will  require  greater  understanding  of  data 
mining.  That  understanding  is  not  planned  in  the  initial  implementation  of  the  AMP-Rev  tool 
and,  therefore,  must  come  from  a  knowledgeable  operator  using  the  system. 
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Pilot  Test  of  Proposed  AMP-Rev 

An  initial  version  of  the  AMP-Rev  will  be  used  to  explore  the  potential  outcome  and 
benefits  of  this  plan.  The  additional  data  proposed  earlier  in  this  section  will  be  incorporated 
into  the  tool  along  with  the  most  recent  2010  perfonnance  review  results.  With  a  total  of  three 
years  worth  of  data,  possible  time-series  trends  may  be  able  to  be  identified.  The  additional  data 
will  be  used  to  expand  the  cross-sectional  analysis  to  investigate  a  wider  possible  range  of 
variations  across  the  geographically  dispersed  pay  pool  locations.  Results  from  using  the  AMP- 
Rev  will  be  presented  to  the  overall  command’s  senior  leadership  for  consideration  of  any 
interventions  that  may  need  to  take  place.  An  action  plan  will  be  developed,  with  the  senior 
leadership,  for  corrective  guidance  to  the  pay  pools  to  improve  consistency  as  well  as  the  use  of 
the  AMP-Rev  tool  in  the  future. 
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CHAPTER  3 
LITERATURE  REVIEW 


Overall,  the  “trend  is  toward  greater  use  of  pay-for-perfonnance  systems.  A  survey  of 

Fortune  1000  firms  found  that  over  80%  of  the  firms  us  some  types  of  performance-based 

compensation  plans.”  (Mathis  &  Jackson,  2008,  pg.  362)  Pay-for-perfonnance  systems  are 

designed  to  positively  motivate  the  workforce  to  perform  their  jobs  better.  Those  that  are 

successful  should  be  encouraged  to  perfonn  even  better;  and  lower  performers  should  be 

identified  and  coached  for  improvement.  Research  has  been  performed  to  better  understand 

employee  motivation  in  general  and  how  best  to  promote  that  motivation  through  effective  pay- 

for-perfonnance  systems. 

Motivation  and  the  Psychological  Contract 

“A  psychological  contract  may  be  defined  as  an  unwritten  agreement  between  individuals 

and  the  organization  of  which  they  are  members.”  (Brown  &  Harvey,  2006,  pg  14)  These 

contracts  “are  highly  subjective  and  are  specific  to  each  employee.  They  concern  both  concrete 

(pay,  working  conditions)  and  abstract  (security,  challenge)  elements  of  the  give-and-take 

between  employer  and  employee”  (Guzzo  &  Noonan,  1994,  pg.  448).  One  study  found  that  when 

a  positive  psychological  contract  was  in  place,  one  that  included  loyalty  and  stability  for  both 

parties,  it  correlated  to  more  positive  employee  work  attitudes  including  identification  with  the 

organization,  increased  devotion,  and  overall  work  satisfaction  (Lee  &  Lui,  2009,  pg.  322). 

When  expectations  are  better  known  between  the  employee  and  their  organization,  that 

psychological  contract  should  improve  employee  morale  and  motivation. 

As  part  of  that  psychological  contract,  some  studies  have  found  differences  on  how 

employees  and  human  resource  departments  characterize  worker  inputs.  Human  resource 

departments  typically  consider  an  employee’s  intelligence,  education,  and  experience  as  traits 
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acquired  when  hiring;  while  in  a  study  employees  viewed  the  three  attributes  as  being  used  every 
day  and  possibly  needing  an  ongoing  investment  (Siegel,  Schraeder,  &  Morrison,  2008,  pg.  71). 
Understanding  which  factors  that  are  important  to  employee’s  motivation  can  help  organizations 
better  formulate  and  even  articulate  their  expectations  to  their  employees  and  build  a  stronger 
psychological  contract  that  can  lead  to  better  work  satisfaction. 

Herzberg  Two-Factor  Theory 

Focusing  on  motivating  an  employee,  their  portion  of  the  psychological  contract  typically 
includes  similar  expectations  of  their  employer.  Frederick  Herzberg  poised  that  employee 
satisfaction  and  dissatisfaction  in  the  job  comes  from  two  entirely  different  set  of  factors. 
Herzberg  describes  these  factors  in  the  following  way: 

What  makes  people  satisfied  at  work  are  factors  that  relate  to  the  content 
of  their  jobs — specifically,  achievement,  recognition  for  achievement, 
interesting  work,  increased  responsibility,  growth,  and  advancement.  On 
the  other  hand,  what  makes  people  unhappy  at  work  is  not  what  they  do 
but  how  well  (or  poorly)  they  are  treated. . . .  The  main  factors  in  this 
group  are  company  policy  and  administration  practices,  supervision, 
interpersonal  relationships,  working  conditions,  salary,  status,  and 
security.  (Herzberg,  1974,  pg.  18) 

Herzberg  graphed  the  most  important  factors  that  both  motivate  and  dissatisfy  employees, 
ranking  them  with  their  level  of  frequency  reported  by  employees;  Table  3  provides  that  ordered 
list.  Each  of  these  factors  may  have  similar  importance,  but  their  ranking  within  the  list  is  due  to 
the  frequency  employees  mentioned  that  factor  not  necessarily  about  how  vital  it  was  to  their 
motivation.  For  instance,  opportunities  for  advancement  and  growth  may  only  be  considered 
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occasionally  by  an  employee,  yet  every  day  that  employee  may  strive  to  achieve  as  well  as  hope 
to  receive  recognition  of  that  achievement.  This  may  be  the  reason  achievement  is  more 
frequently  listed  as  a  motivator  by  the  employee  than  the  factors  of  advancement  and  growth. 

Table  3:  Herzberg  Factors  for  employee  satisfaction  and  dissatisfaction 


Most  Frequent  Motivating/Satisfier  Factors 

Most  Frequent  Hygiene/Dissatisfier  Factors 

1 .  Achievement 

1 .  Company  policy  and  administration 

2.  Recognition  for  achievement 

2.  Supervision 

3.  Work  itself 

3.  Interpersonal  relations 

4.  Responsibility 

4.  Working  conditions 

5.  Advancement 

5.  Salary 

6.  Growth 

6.  Status 

7.  Security 

Table  3  provides  an  ordered  list  for  a  typical  organization;  this  order  may  be  very 
different  due  to  either  employee  or  organizational  culture.  In  one  study,  salary  was  the  greatest 
factor  in  dissatisfaction,  as  indicated  by  87  percent  responses  from  employees,  while  recognition 
of  achievement  the  5th  most  frequent  satisfier  (Sledge,  Miles  &  Coppage,  2008,  pg  1678).  A 
separate  study  “indicated  that  the  respondents  were  least  satisfied  with  their  salary  as  well  as 
organizational  policies  and  administration”  (Schroder,  2008,  pg  239). 

Salary,  in  particular,  was  a  special  case  in  Herzberg’s  study:  “Because  of  its  ubiquitous 
nature,  salary  commonly  shows  up  as  motivator  as  well  as  hygiene.  Although  primarily  a 
hygiene  factor,  it  also  often  takes  on  some  of  the  properties  of  a  motivator  with  dynamics  similar 
to  recognition  of  achievement.”  (Herzberg,  1974,  pg.  20)  As  mentioned  earlier,  salary  increase 
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is  expected  to  be  the  most  influential  aspect  affecting  the  employee  by  a  pay-for-performance 
system.  Although  salary  is  typically  considered  a  dissatisfaction  or  hygiene  factor,  it  could  also 
affect  several  employees  in  a  positive  manner  and  increase  motivation.  This  duality  that  salary 
can  be  either  a  motivational  or  dissatisfaction  factor  could  be  the  impetus  for  creating  a  pay-for- 
performance  system;  to  properly  motivate  as  both  an  incentive  and/or  punishment  for  an 
employee’s  perfonnance. 

A  pay-for-performance  system  can  impact  up  to  four  dissatisfaction  factors.  That  system 
is  a  company  policy  that  is  commonly  administered  by  supervisors.  It  also  directly  affects  one’s 
salary  and  is  likely  to  play  a  factor  in  interpersonal  relations  when  everyone  is  discussing 
preparation  for  and  results  from  the  performance  review.  As  a  motivator,  the  pay-for- 
performance  system  is  primarily  tied  to  recognition  of  achievement.  In  a  few  organizations, 
those  results  may  be  used  to  investigate  how  best  to  grow  an  employee  or  consider  him/her  for 
advancement.  Since  most  of  the  factors  that  the  pay-for-perfonnance  system  influences  are 
dissatisfiers,  many  will  naturally  oppose  the  use  of  any  system  since  it  is  more  likely  to  lead  to 
dissatisfaction  than  as  a  motivational  tool.  After  identifying  the  more  important  factors  that  may 
motivate  or  more  likely  demotivate  an  employee,  the  next  topic  discusses  how  employees 
typically  determine  if  their  situation  is  fair  and  equitable. 

Equity  Theory 

“According  to  equity  theory,  employee  satisfaction  and  motivation  depend  on  how  fairly 
the  employees  believe  they  are  treated  in  comparison  to  peers.”  (Dubrin,  2010,  pg  298) 
Individuals  consider  their  level  of  input  (work  effort,  job  qualifications,  skills  and  experience, 
and  support  of  coworkers)  in  relation  to  their  received  outcome  (including  salary,  status  and 
recognition,  and  other  benefits)  against  some  comparison  group  (Dubrin,  2010,  pg  298).  This 
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relationship  is  based  solely  on  the  individual’s  perception  and  does  not  necessarily  have  to 
coincide  with  reality.  This  perceived  equity  ratio  is  expressed  below. 


Who  the  individual  compares  to  is  also  subjective;  it  could  be  someone  working  at  the 
same  level  within  the  office,  a  team  from  a  different  functional  division,  or  the  individual  could 
compare  his/her  current  situation  with  a  job  held  several  months  earlier.  When  the  perceived 
comparison  appears  inequitable,  there  will  be  tension  to  change  the  conditions  of  the 
outcome/input  expression.  That  tension  will  be  directly  proportionate  to  the  level  of  perceived 
inequity  (Adams,  1963,  pg.  427),  the  greater  the  perceived  inequality  the  stronger  the  urge  to 
alter  the  conditions.  Adams  listed  several  options  employees  are  likely  to  do  to  reduce  the 
feeling  of  inequity  (1963,  pg.  427-429): 

1 .  They  may  change  their  level  of  input,  for  instance  they  may  work  less  to  offset  a 
feeling  that  they  are  not  being  paid  enough.  On  the  other  hand,  employees  are  likely 
to  step  up  their  efforts  if  they  notice  the  team  around  them  is  working  at  a  higher 
level. 

2.  They  may  attempt  to  change  their  outcomes  like  negotiating  for  a  raise  by  presenting 
their  high  level  of  inputs  (amount  they  work,  superior  skills  or  experience,  etc). 

3.  They  may  alter  their  perceptions  of  their  own  inputs  and  outcomes;  reconsidering  the 
value  of  the  inputs  they  provide  and  the  current  resultant  outcomes. 

4.  They  may  reconsider  or  influence  other’s  inputs  and  outcomes;  by  encouraging  the 
comparison  person  to  change  their  level  of  effort  or  even  leave  the  workforce,  their 
perceived  inequity  could  rebalance. 
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5.  They  may  change  who  they  compare  themselves  to;  instead  of  comparing  themselves 
with  someone  that  received  a  very  large  raise  they  may  now  consider  their  inputs  with 
someone  else  who  had  a  similar  raise. 

6.  They  may  remove  themselves  from  the  situation  through  higher  absenteeism, 
requesting  transfer  to  another  group,  or  even  finding  another  job  entirely. 

Any  of  the  above  mechanisms  may  be  used  by  the  employee  to  relieve  any  perceived 
inequity;  in  fact,  employees  may  use  several  of  the  above  options  automatically  without 
realizing.  In  one  experiment,  it  was  found  that  when  an  employee  was  informed  that  his/her 
qualifications  (inputs)  were  not  up  to  standard,  but  they  would  still  be  compensated  at  a  higher 
pay  (outcome);  that  employee  perfonned  at  a  higher  level  than  those  that  were  appropriately  paid 
for  their  qualifications.  (Adams  &  Jacobsen,  1964,  pg.  23).  This  higher  level  of  performance 
was  considered  due  to  the  perceived  inequity  of  the  employee;  the  worker  felt  overcompensated 
for  his/her  qualifications  and  therefore  stepped  up  his/her  input  to  compensate.  However,  if  a 
group  of  individuals  are  overcompensated  collectively,  they  are  likely  to  compare  their  input  and 
outcomes  with  those  around  them  who  are  equally  overcompensated.  This  condition  will  likely 
not  lead  to  any  improvement  in  perfonnance  since  the  employees  will  believe  their  current 
situation  is  equitable  (Adams,  1968,  pg.  315). 

Performance  Management 

“An  effective  performance  management  system  should  do  the  following:  make  clear  what 

the  organization  expects,  provide  performance  infonnation  to  employees,  identify  areas  of 

success  and  needed  development,  and  document  perfonnance  for  personal  records.”  (Mathis  & 

Jackson,  2008,  pg.  327)  The  many  governmental  appraisal  systems,  to  include  NSPS,  the  Total 

Anny  Perfonnance  Evaluation  System  (TAPES),  and  the  Contribution-based  Compensation  and 

Appraisal  System  (CCAS),  all  were  designed  to  perform  the  four  elements  above.  Yet  an 
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emphasis  on  pay-for-performance  caused  the  development  of  these  newer  systems,  such  as  NSPS 
and  CCAS,  to  augment  TAPES  that  typically  was  not  used  to  link  pay  with  perfonnance. 

Perfonnance  appraisals  can  be  used  for  two  competing  aspects:  one  can  focus  on  how 
best  to  develop  an  employee  and  another  is  to  assist  in  administration  tasks  like  detennining  pay 
raises,  promotion  opportunities,  and  disciplinary  actions  (Mathis  &  Jackson,  2008,  pg.  333).  All 
these  actions  are  important  aspects  of  perfonnance  management  and,  thus,  better  emphasizing  of 
three  separate  review  sessions  should  be  considered.  Mathis  and  Jackson  (2008)  discuss  how 
some  employers  first  discuss  overall  performance,  reviewing  the  positive  contributions  and  areas 
of  improvement.  Later,  a  separate  review  will  discuss  training  and  development  opportunities 
that  may  better  reinforce  the  employee’s  contributions  or  address  a  performance  gap.  Finally,  a 
third  discussion  would  focus  on  compensation,  including  pay  raises  and  promotion  opportunities. 
In  many  annual  appraisals,  particularly  pay-for-performance,  a  single  review  with  the  supervisor 
will  quickly  outline  both  performance  and  the  resulting  pay  raise  and  bonuses.  Most  employees 
will  then  focus  on  the  pay  increase,  and  the  performance  discussion  may  be  lost.  The 
development  opportunity  discussion  many  times  never  occurs;  either  due  to  lack  of  resources, 
little  planning  to  provide  employee  training,  or  it  may  be  expected  that  the  employee  will  initiate 
a  development  discussion. 

Blimes  and  Gould  (2009)  state  several  issues  with  government  perfonnance  appraisal 
systems,  including  the  difficulty  in  developing  objectives  since  many  government  outcomes 
cannot  be  measured  easily.  Advancing  trade,  reducing  poverty,  and  many  other  activities  vital  to 
government  cannot  pinpoint  success  within  an  annual  cycle  easily.  Few  employees  trust  that 
they  will  be  accurately  evaluated  since  most  supervisors  have  inadequate  training  and  were  not 
required  to  meet  concrete  standards  themselves  (Blimes  &  Gould,  2009,  pg.  199). 
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Performance  Rating  Systems 

Most  governmental  rating  systems,  including  TAPES,  CCAS,  and  NSPS,  apply  a  graphic 
rating  scale  to  score  each  employee  using  numerical  representation  of  descriptions  of 
perfonnance.  These  descriptions  include  role  model,  high  performance,  valued  employee,  or 
need  improvement  to  assist  the  rater  in  designating  an  employee’s  performance.  The  benefit 
with  a  performance  rating  system  is  that  it  is  easy  to  develop  and  provide  a  simple  rating  process. 
This  system  can  work  very  well  when  the  categories  being  scored  are  linked  closely  with  the 
employee’s  actual  job  duties  (Mathis  &  Jackson,  2008). 

When  attempting  to  compare  employees  to  each  other,  there  is  a  benefit  of  having  more 
generic  objectives.  A  generic  objective  like  “employee  development”  is  broad  enough  to  include 
achievements  of  some  employees  pursuing  academic  degrees  while  others  can  focus  on  technical 
training.  With  these  generic  goals,  the  rater  may  find  it  more  difficult  to  detennine  superior 
perfonnance  with  broad  objectives  than  employee-specific  goals  that  spell  out  exactly  what  it 
takes  to  achieve  and  surpass  that  objective.  “Another  drawback  is  that  the  descriptive  words 
sometimes  used  in  scales  may  have  different  meaning  to  different  raters”  (Mathis  &  Jackson, 
2008,  pg.  342).  What  “exceeds  expectations”  for  one  supervisor  may  not  even  meet  the  bar  for 
another. 

The  drawback  of  different  meanings  for  objectives  and  scoring  is  compensated  for  by  the 

NSPS  use  of  pay  pools.  Since  a  body  of  individuals  must  review  over  a  large  group  of 

employees,  the  group  becomes  a  normalization  factor  to  ensure  a  common  standard,  and 

common  scoring  has  occurred.  This  information  defining  objectives  and  scorings  must  then  be 

communicated  to  supervisors  and  employees  so  that  the  entire  organization  knows  what  to 

expect  from  objectives  and  how  to  achieve  desired  scores.  Essentially,  this  communication  will 

ensure  a  correct  psychological  contract — the  “unwritten  agreement  between  and  employee  and 
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the  organization  which  specifies  what  each  expects  to  give  to  and  receive  from  the  other” 
(Ivancevich,  Konopaske,  &  Matteson,  2008,  pg  129).  Jeffery  Irnmelt,  CEO  of  General  Electric 
was  quoted  as  saying,  “We  give  feedback,  we  coach,  we  invest  in  training — and  we  have  clear 
perfonnance  goals.  People  know  where  they  stand;  they  know  we  will  help  them  to  be  the  best 
they  can  be;  and  they  agree  on  their  annual  goals  and  objectives”  (Bartlett  and  McLean,  2005, 
pg.  12).  The  key  to  a  performance  rating  system  is  the  initial  involvement  of  the  employee  in  the 
goals  and  objectives,  providing  continual  feedback,  and  properly  coaching  each  employee  to 
become  better. 

Forced  Ranking  Systems 

“While  traditional  performance  appraisals  tend  be  criterion  based  (setting  a  performance 
bar),  forced  ranking  is  about  distinguishing  people.  Forced  ranking  demands  a  differentiation 
among  performers”  (Hazels  &  Sasse,  2008,  pg.  35).  Bartlett  and  McLean  (2003)  summarized  the 
mechanics  of  forced  ranking  that  Jack  Welsh  ingrained  into  General  Electric’s  organizational 
culture.  Ranking  would  compel  supervisors  to  distinguish  the  top  20  percent  of  their  workforce 
as  top  performers,  who  would  be  most  rewarded  to  ensure  they  remained  with  the  company.  The 
lowest  10  percent  would  also  be  identified  who  would  then  be  coached,  trained,  and  later 
tenninated  if  their  performance  did  not  improve.  GE’s  ranking  looked  not  only  at  performance, 
but  also  at  the  promotability  of  each  employee.  An  employee  who  was  a  high  perfonner,  but 
was  deemed  unlikely  to  be  promoted  would  be  highly  valued  but  not  as  prized  as  someone  who 
exhibited  both  high  perfonnance  and  high  ability  to  be  promoted. 

Grote  (2005,  as  cited  by  Hazels  &  Sasse,  2008,  pg.  36)  claimed  the  benefits  of  this 
system:  “Without  forced  ranking,  managers  are  tempted  to  be  too  kind  and  not  confront 
perfonnance  problems.  Furthermore,  it  may  be  difficult  for  a  manager  to  admit  he  or  she  has 
made  a  hiring  mistake.”  It  also  allows  a  company  to  focus  limited  resources  for  bonuses, 
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training,  and  awards  appropriately  to  the  higher  ranked  perfonners.  Due  to  these  advantages,  “A 
recent  study  by  Novations  Group  Inc.  found  that  a  total  of  54.8%  of  all  responding  companies 
used  forced  ranking”  (Hazels  &  Sasse,  2008,  pg.  35). 

Hazels  and  Sasse  (2008)  lists  several  drawbacks  of  forced  ranking;  the  most  significant 
can  cause  negative  effects  on  morale,  since  employees  are  constantly  concerned  about  their 
ranked  order.  “The  Novations'  study  cited  earlier  found  that  almost  44  percent  of  respondents  felt 
morale  was  negatively  affected”  (pg.  37).  Forced  ranking  can  also  eliminate  teamwork  since 
workers,  in  fear  of  losing  their  own  ranking,  will  be  reluctant  to  help  others.  There  are  concerns 
that  the  lowest  ranked  employee  in  a  high-performing  team  may  actually  be  a  very  high 
perfonner  as  measured  against  standards  and  therefore  penalized  for  working  in  such  a  strong 
team.  The  workplace  can  also  become  more  political  as  individuals  vie  for  the  jobs  that  are  most 
likely  to  yield  the  best  ranking.  Even  proponents  of  force  ranking  typically  admit  that  over  time, 
there  is  a  diminishing  return  as  the  lowest  perfonners  have  already  been  weeded  out. 

In  a  one  study,  certain  aspects  of  forced  ranking  appealed  to  potential  employees  more 
than  others.  Employee  candidates  “were  most  inclined  to  find  attractive  those  systems  that  had 
less  stringent  consequences  for  low  perfonners,  higher  differentiation  of  rewards,  large 
comparison  groups,  and  frequent  feedback.  The  consequences  for  low  performers  had  the  single 
most  powerful  influence  on  their  attraction  to  different  FDS  [Forced  Distribution  Systems]” 
(Blume,  Baldwin,  &  Rubin,  2009,  pg.  86).  Therefore,  the  policy  on  what  occurs  to  lower 
perfonners  will  be  the  most  influential  to  the  organization’s  culture  that  uses  a  forced  ranking 
system.  Therefore,  Hazels  and  Sasse  (2008),  recommend  only  using  forced  distribution  “in 
conjunction  with,  not  in  place  of,  traditional  performance  evaluations”  and  once  “it  has  been 
used  to  eliminate  those  individuals  not  doing  their  jobs  at  acceptable  levels,  it  should  be 
discontinued”  (pg.  37). 
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Government  Accountability  Office  (GAO)  Recommendations  for  NSPS 

In  2008,  the  GAO  made  the  following  four  recommendations  on  how  the  Department  of 
Defense  could  improve  their  NSPS  implementation  (Ferrell,  2008,  pg.  42): 

•  Require  a  third  party  to  perform  predecisional  demographic  and  other 
analysis  as  appropriate  for  pay  pools. 

•  Require  commands  to  publish  the  final  overall  rating  results. 

•  Provide  guidance  to  pay  pools  and  supervisors  that  encourages  them  to 
rate  employees  appropriately,  including  using  all  categories  of  ratings 
as  warranted  by  comparing  employees’  individual  perfonnance  against 
the  standards. 

•  Develop  and  implement  a  specific  action  plan  to  address  employee 
perceptions  of  NSPS. 

The  original  intent  of  the  AMP-Rev  tool  addresses  GAO’s  first  recommendation  with  the 
exception  that  AMP-Rev’s  suggested  use  was  to  be  post-decisional.  That  way  AMP-Rev  would 
provide  analysis  on  trends  and  demographics  that  could  be  used  to  guide  policy  the  next  year 
without  any  fear  that  it  could  influence  the  results  of  the  current  year’s  results.  AMP-Rev  could 
be  used  by  a  third  party  before  a  decision  of  the  results  was  finalized,  but  that  would  be  a  more 
aggressive  use  of  the  tool  than  was  originally  intended.  This  tool  can  also  be  used  to  address 
GAO’s  second  recommendation  by  providing  a  means  to  publish  more  than  just  rating  results, 
adding  share  percentage  and  payout  information  as  well,  that  would  increase  transparency 
without  jeopardizing  personal  infonnation.  These  results  were  designed  to  be  used  to  provide 
guidance  to  pay  pools  and  supervisors  (3rd  GAO  recommendation)  and  could  become  a  vital  part 
of  a  specific  action  plan  to  address  employee  concerns  with  NSPS  (4th  GAO  recommendation). 
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Over  a  year  later,  the  most  recent  GAO  report  included  three  recommendations  for  action 
(Ferrell,  2009,  pg.  50): 

•  Review  and  evaluate  the  effectiveness  of  the  department’s  training. 

•  Ensure  that  guidance  is  in  place  for  conducting  a  postdecisional 
analysis  that  specifies  what  process  the  components  should  follow 
to  investigate  and  eliminate  potential  barriers  to  fair  and  equitable 
ratings. 

•  Include,  as  part  of  the  department’s  monitoring  of  the 
implementation  of  its  system,  efforts  to  monitor  and  evaluate  how 
the  safeguards  specifically  are  implemented  by  lower-level 
organizations  across  the  department. 

The  AMP-Rev  tool  was  designed  to  address  the  last  two  GAO  recommendations.  Note 
that  the  second  recommendation  is  similar  to  2008  with  two  major  differences.  First,  third  party 
analysis  is  no  longer  recommended,  meaning  the  data  reduction  can  be  perfonned  as  an  internal 
measure  within  a  command.  Second,  the  recommendation  switches  from  pre-decisional  to  post¬ 
decisional  analysis.  This  mimics  the  original  intent  of  AMP-Rev,  to  be  used  within  a  command 
for  post-decision  analysis  to  monitor  and  ensure  fair  practices  in  perfonnance  scoring  has 
occurred  (3  d  GAO  recommendation).  Therefore,  the  prototype  of  AMP-Rev  addresses  the 
primary  concerns  GAO  has  with  the  current  implementation  of  NSPS. 
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CHAPTER  4 
RESULTS 


In  January  2010,  the  eight  geographical  locations  within  the  studied  command  completed 
another  performance  review  involving  over  1,800  employees.  This  additional  data  was 
incorporated  within  the  prototype  AMP-Rev  tool  to  provide  additional  time  series  analysis.  The 
additional  data  for  each  employee  mimicked  the  categories  described  in  Table  1  within  Part  A. 
Most  of  the  proposed  supplementary  data  for  each  employee  was  not  included  due  to  a  concern 
for  disseminating  employee  personal  infonnation:  the  employee’s  performance  score, 
performance  payout,  current  salary,  and  pro-rated  salary  increase.  For  very  large  locations  that 
broke  their  employees  into  groups  to  be  evaluated  in  separate  pay  pools,  the  employee  was 
matched  to  his/her  particular  pay  pool.  This  allowed  for  a  check  on  internal  consistency  to 
ensure  that  within  one  geographical  location  all  of  their  pay  pools  acted  consistently  with  each 
other. 

Additional  Data  Including  Employee  Financial  Benefits  for  Performance 

Avoiding  the  release  of  personal  information  for  each  employee,  summary  data  instead 
was  provided  on  resulting  performance  payout  and  awards  for  each  pay  pool.  The  following  data 
was  most  valuable  to  the  cross-sectional  analysis  between  pay  pools:  average  payout  %,  average 
salary  increase  %,  and  average  bonus  %.  These  percentage  numbers  were  used  to  nonnalize 
performance  payouts  across  a  wide  range  of  employee  salaries.  Based  upon  the  amount  of  shares 
awarded,  each  employee  was  designated  an  amount  for  his/her  payout,  which  included  both  a 
salary  increase  and  a  one-time  bonus  award.  For  example,  one  employee  initially  making 
$50,000  a  year  may  receive  a  $2,500  payout,  of  which  $2,000  goes  to  an  increase  in  annual 
salary  and  $500  to  a  one-time  bonus.  A  supervisor,  within  the  same  pay  pool,  may  make 


$100,000  a  year  and  received  a  $5,000  payout  broken  into  a  $2,000  increase  in  salary  and  a  one- 
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time  bonus  of  $3,000.  Although  the  payout  dollars  between  the  employee  and  the  supervisor  are 
remarkably  different,  they  both  received  the  same  payout  percentage  as  indicated  below. 

-  -  -  5% 

Looking  at  the  distribution  of  the  payout,  both  the  employee  and  the  supervisor  received 
the  same  salary  increase:  $2,000.  However,  the  employee  had  a  double  the  salary  increase  % 
than  the  supervisor  as  shown  in  the  inequality  below. 


Averaging  the  two  salary  increase  %  above  (4  percent  and  2  percent)  would  yield  3 
percent  overall.  The  bonus  %  also  uses  each  employee’s  current  salary  as  a  nonnalization  factor. 
In  this  case,  the  supervisor  has  a  much  higher  bonus  %  than  the  employee,  yet  the  two  calculates 
an  average  bonus  %  of  2%. 


This  averaging  of  salary  increase  and  bonus  was  perfonned  for  all  the  employees  within 
each  of  the  pay  polls.  These  measures  provided  a  nonnalized  comparison  between  pay  pools  on 
the  primary  pay-for-performance  factor:  employee  salary.  This  information  led  to  nearly  half  of 
the  findings  relating  to  the  consistency  of  the  overall  command’s  implementation  of  their  pay- 
for-performance  implementation. 


34 


Overall  Results  Oppose  Hypothesis  of  Expected  Inconsistency  Across  Command 

For  this  command,  the  analysis  did  not  support  the  hypothesis  that  several  inconsistencies 
within  the  pay-for-performance  system  would  be  identified  and  need  to  be  prioritized  for 
improvement.  Instead,  this  command,  even  with  the  eight  geographically  dispersed  locations, 
had  a  high  level  of  consistency  for  salary  (prime  dissatisfier  factor)  and  ratings  (secondary 
satisfier  factor).  There  were  a  couple  areas  of  potential  improvement,  which  are  identified  in  the 
findings  that  could  further  increase  consistency  with  the  ratings  and  payouts.  Each  finding  is 
categorized  in  one  of  three  areas:  findings  that  illustrate  a  positive  consistency  of  results  across 
the  command,  findings  that  indicate  a  potential  area  of  improvement  and  could  be  used  to 
stimulate  corrective  action,  and  findings  that  point  to  a  potential  trend  that  should  be  further 
investigated  but  require  no  corrective  action  at  this  time. 

Finding  1:  Performance  Ratings  Increasing  Over  Time  (Potential  Area  of  Improvement) 

Over  the  last  two  years,  the  overall  command’s  average  rating  has  increased  about  0.07 
points  each  year.  Although  a  relatively  modest  creep,  this  increase  in  rating  performance  is 
consistent  across  most  of  the  geographical  locations.  Figure  5  shows  only  one  exception  to  this 
rating  creep:  pay  pool  6  (PP6)  average  rating  went  down  between  2009  and  2010.  In  all  other 
cases,  the  average  performance  rating  increased  every  year  for  every  pay  pool.  In  many  cases 
these  increases  were  very  small;  however,  both  pay  pool  4  and  5  provided  ratings  improvements 
of  about  a  half  a  point  each.  Such  a  large  increase  in  performance  rating  would  result  in  an 
additional  share  value  for  every  employee  within  that  pay  pool. 
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Figure  5:  Rating  Creep  Across  All  Pay  Pools 
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The  small  increase  in  performance  ratings  across  all  locations  may  indicate  the  pay  pools’ 
desire  to  increase  the  employee  satisfier  factor  of  recognition  through  slightly  elevated  ratings. 

If  this  performance  creep  is  consistent  across  all  pay  pools,  a  short-term  positive  effect  on 
employee  satisfaction  is  likely.  However,  over  time,  elevated  ratings  will  become  expected  and 
there  will  no  longer  be  room  on  the  performance  rating  scale  to  recognize  high  performers.  In 
2010,  pay  pools  4  and  5  provided  high  average  ratings  of  4.2  and  3.9  respectively  indicate  that 
their  average  employee  “exceeds  expectations.”  Those  pay  pools  may  find  it  difficult  to 
recognize  a  higher-performing  employee  in  subsequent  years  since  the  rating  scale  only  goes  up 
to  5.  One  of  the  purposes  of  NSPS  was  to  provide  a  scale  that  would  recognize  a  range  of 
perfonnance;  that  benefit  will  be  nullified  when  everyone  who  previously  was  considered  a 
valued  employee  (performance  rating  of  3)  over  time  has  an  expected  rating  of  a  role  model,  5, 
due  to  this  performance  rating  creep.  A  potential  area  of  improvement  may  be  for  the  command 
to  provide  guidance  to  maintain  an  average  performance  rating  around  3.5  or  lower. 
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Finding  2:  Payout  for  Employees  has  Decreased  Significantly  Each  Year  (Potential  Trend) 

NSPS  was  designed  so  that  the  more  shares  given  across  a  particular  pay  pool,  the  lower 
each  individual  share  is  worth.  Figure  6  shows  this  relationship.  Each  dot  indicates  an  individual 
pay  pool,  those  locations  with  multiple  pay  pools  have  additional  data  points  plotted  to  this 
graph.  Each  pay  pool  has  its  average  number  of  shares  awarded  to  its  employees  on  the 
horizontal  axis  and  the  resulting  share  value  on  the  vertical  axis.  A  linear  trendline  shows  for 
each  year  as  more  shares  are  awarded  the  lower  that  pay  pool’s  share  value. 

The  data  also  shows  that  for  each  year,  significantly  fewer  amount  of  funds  have  been 
placed  within  the  pay  pool.  The  share  value  from  2008  is  double  that  of  2010.  The  amount  of 
money  placed  into  each  pay  pool  is  established  at  a  higher  headquarters;  this  command  and  its 
subsequent  pay  pools  implemented  what  had  been  dictated  as  the  amount  in  pay  pools. 
Nonetheless,  the  total  amount  of  dollars  for  the  pay  pools  to  be  distributed  has  lowered  resulting 
employees  with  a  higher  number  of  shares  being  more  financially  rewarded  in  2008  than  if  they 
received  the  same  number  of  shares  in  2009,  or  2010. 

Figure  6:  Each  Year  Has  Provided  Less  Funding  for  Pay  Pools 
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Finding  3:  Average  Employee  Payout  Consistent  Across  Pay  Pools  (Positive  Consistency) 

The  share  value  balances  the  pay  pool’s  differences  in  providing  perfonnance  ratings. 
Figure  7  illustrates  the  consistency  of  average  payout  %  regardless  of  the  number  of  shares 
awarded  (left  graph)  or  geographic  location  (right  graph).  For  a  given  year,  all  data  points  create 
a  relatively  flat  line,  showing  that  all  pay  pools  awarded  very  consistent  payouts.  Reviewing 
Figure  7  for  2010,  one  pay  pool  awarded  an  average  of  3.57  shares  while  the  other  extreme  was 
only  2.06  shares.  However,  the  average  employee  would  have  made  a  slightly  higher  payout 
working  for  the  pay  pool  that  gave  fewer  shares. 

This  paper’s  hypothesis  expected  inconsistency  between  pay  pools;  however,  as  Figure  7 
shows,  even  with  deviations  in  the  performance  rating  and  number  of  shares  awarded,  the 
employee  payout  was  relatively  consistent  across  all  pay  pools.  This  balancing  factor  can  be  best 
seen  by  first  observing  the  small  dip  in  the  payout  of  pay  pool  4  (PP4)  in  both  2009  and  2010  in 
Figure  7.  In  contrast,  Figure  5  shows  that  PP4  generally  awarded  higher  performance  ratings. 
Although  in  2010,  PP4  had  by  far  the  highest  average  performance  rating,  it  also  had  the  lowest 
payout  %  of  all  other  pay  pools.  A  final  observation  can  be  seen  confirming  finding  2;  for  each 
successive  year  the  average  payout  to  employees  decreases  significantly. 

Figure  7:  Average  Employee  Payout  Consistent  Regardless  Of  Shares  Awarded 
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Finding  4:  Average  Salary  Increase  was  More  Consistent  in  2010  (Positive  Consistency) 

Although  consistency  of  average  payout  is  a  powerful  indicator,  the  most  desired  result 
for  the  majority  of  employees  is  getting  the  highest  salary  increase  %  possible.  As  the  primary 
dissatislicr  factor,  Figure  8  maps  out  the  average  salary  increase  %  across  pay  pools.  Although 
there  is  a  much  greater  variability  in  salary  increase  %  than  total  payout  %,  in  2010,  consistency 
improved  considerably  across  the  pay  pools.  The  large  variability  in  2009  in  salary  increase  % 
is  likely  due  to  first  time  implementations  of  NSPS  for  five  of  the  eight  pay  pools.  The  improved 
consistency  between  2009  and  2010  can  be  best  illustrated  by  comparing  the  difference  between 
the  lowest  (PP2,  PP3)  and  highest  (PP7)  salary  increase  pay  pools.  This  difference  between  the 
two  extreme  pay  pools  would  result  in  an  employee  who  makes  $50,000  a  year  to  receive  $1,090 
greater  salary  increase  in  2009,  but  only  $300  greater  salary  increase  in  2010.  Maintaining  and 
even  narrowing  the  gap  for  salary  increase  between  pay  pools  is  probably  the  most  important 
factor  to  illustrate  pay-for-performance  consistency  across  the  command. 

Figure  8:  Average  Salary  Increase  Was  More  Consistent  in  2010 


Figure  8  also  highlights  finding  2  showing  that  for  each  successive  year,  employees 
receive  smaller  salary  increases.  The  previously  used  pay  system,  general  schedule  (GS), 
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typically  employed  automatic  salary  increases  based  on  the  amount  of  time  an  employee  worked 
within  a  particular  pay  grade;  those  salary  increases  are  highlighted  on  Figure  8  with  a  series  of 
dashed  gray  lines.  The  top  line  of  3.33  percent  annual  salary  increase  was  only  given  to  the 
employees  just  beginning  within  the  band  (Step  1  employees);  that  high  salary  increase  was 
automatically  given  for  at  most  two  years.  The  middle  line  (Step  5)  indicates  employees  that 
may  get  a  raise  every  two  years  and  the  lowest  line  (Step  9)  was  for  the  most  senior  employees 
within  a  band.  Most  employees  fall  within  the  lower  two  gray  lines  gaining  an  average  annual 
pay  raise  of  around  1  percent. 

In  2008,  every  employee,  including  the  most  senior,  was  likely  to  get  much  larger  raises 
than  expected  under  the  GS  system.  This  likely  resulted  in  overall  high  employee  satisfaction 
with  NSPS  for  that  first  year.  However,  such  large  salary  increases  over  time  would  drive 
everyone’s  salary  quickly  toward  the  cap  for  a  given  pay  band.  At  that  point,  there  would  be  no 
more  room  to  give  salary  increases,  and  all  of  the  payout  would  only  go  to  one-time  bonuses. 
Employees  now  at  their  salary  cap  may  have  less  incentive  to  perfonn,  since  their  salary  would 
remain  unchanged  regardless  of  their  perfonnance.  Lowering  the  amount  of  funds  allowed  to  be 
distributed  each  year  in  both  2009  and  2008  may  have  been  a  response  to  bring  salary  increases 
back  to  a  level  that  would  not  ‘’’break  the  bank”  of  the  command.  Even  at  the  lowest  level  in 
2010,  the  average  employee  would  still  have  a  larger  salary  increase  under  NSPS  rather  than  GS. 
If  this  salary  increase  level  goes  lower  than  the  0.8  percent  of  the  GS  Step  9,  it  would  likely 
cause  employees  to  consider  the  NSPS  system  less  advantageous  than  the  automatic  pay  raises 
on  GS.  It  is  recommended  that  the  average  salary  increase  should  be  maintained  between  the 
step  9  (0.08  percent)  and  step  5  (1.4  percent)  levels. 


40 


Finding  5:  Variation  in  Salary  Increase  %  Due  to  Differences  in  Pay  Pool  Salary  Split  % 
(Potential  Area  of  Improvement) 

Investigating  further  into  the  variations  in  salary  increase,  Figure  9  (left  graph)  plots  on 
top  the  average  salary  increase  %  for  2010,  the  same  line  as  in  Figure  8.  The  lower  line  in  Figure 
9  (left)  provides  the  bonus  %.  For  every  pay  pool  in  2010,  they  gave  more  of  the  total  payout  to 
salary  increase  than  in  bonus,  resulting  in  the  salary  line  consistently  above  the  bonus  one.  To 
accentuate  this  difference,  the  middle  line  is  the  average  payout  %  divided  in  half  so  that 
graphically,  it  should  bisect  the  salary  and  bonus  lines.  Although  the  middle  payout  %  line  is 
relatively  flat  and  therefore  more  consistent  across  the  pay  pools,  larger  variations  in  salary  and 
bonus  offset  each  other.  In  particular,  pay  pools  4  and  7  seem  to  have  higher  ratios  of  salary  to 
bonus. 

Figure  9  (right  graph),  graphically  shows  that  for  a  given  year,  as  a  pay  pool  designates  a 
higher  percentage  of  the  total  payout  to  salary,  the  natural  result  is  that  a  higher  employee  salary 
increase  occurs.  If  salary  increase  %  is  the  primary  dissati slier  for  employee  and  consistency  is 
the  goal,  the  key  may  be  to  consider  guidelines  for  pay  pools  to  be  more  consistent  in  designating 
the  percentage  proportion  of  the  total  payout  to  salary.  Since  the  total  payout  is  consistent  across 
all  pay  pools,  finding  3,  then  making  the  share  split  %  more  consistent  should  result  in  less 
variability  in  salary  increase  %. 
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Figure  9:  Average  Salary  Increase  Related  to  Inconsistent  Pay  Pool  Salary  Split  %: 


Finding  6:  Pay  Pool  Variations  in  Share  Split  %  (Potential  Area  of  Improvement) 

Finding  5  recommended  considering  consistency  in  pay  pool  share  split  %  to  reduce 
variability  in  salary  increase  %  across  geographical  locations.  Histograms  categorized  the  share 
split  %  for  each  pay  pool  for  2010,  collectively  illustrated  in  Figure  10.  Each  histogram  is 
broken  down  into  5  percent  bins  starting  with  0  percent,  0.1-5  percent,  5.1-10  percent  ...  95. 1- 
100  percent.  The  height  of  the  blue  lines  indicates  the  relative  number  of  employees  whose 
share  percentage  falls  into  that  particular  bin.  For  instance,  pay  pool  1  has  a  large  variety  of 
share  percentages  awarded;  however  most  are  45  percent  and  above.  Pay  pool  2  has  slightly  less 
variety  and  most  are  65  percent  and  above.  Pay  pools  3  and  7  gave  the  same  share  percentage  to 
every  one  of  their  employees,  82.9  percent  and  69.3  percent  respectively;  while  pay  pool  6 
primarily  gave  their  employees  100  percent  to  salary,  with  only  a  few  exceptions  of  47.3  percent 
and  0  percent.  Greater  consistency  of  share  split  %  may  require  guidance  to  promote 
standardization  across  all  the  pay  pools. 
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Finding  7:  Relationship  Between  Performance  Rating  and  Job  Position  (Potential  Trend) 

NSPS  uses  seven  different  pay  schedules  that  are  loosely  related  to  job  positions.  Refer  to 
Table  2  in  Part  A  for  more  definition.  Figure  1 1  graphs  the  average  performance  rating  (left)  and 
share  split  %  (right)  for  the  different  job  positions.  Figure  1 1  (left)  shows  that  certain  job 
positions  seem  to  receive  higher  perfonnance  ratings.  From  highest  performance  rating  down 
the  following  order  is  observed: 

1 .  Supervisors  (Y C  YF) 

2.  Analysts,  Scientist,  and  Engineers  (YA  YD) 

3.  Administrative  and  Technical  staff(  YB,  YE) 

4.  Students  (YP) 
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Figure  10:  Finding  6:  2010  Variations  in  Share  Split  % 
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Figure  11:  Relationship  between  Rating  and  Pay  Schedule 


This  progression  in  ratings  may  be  due  to  an  employee’s  position  having  more  leverage 


to  directly  affect  a  command’s  objectives.  Students,  being  new  to  the  organization  and  only 


working  part  time,  are  likely  not  to  be  able  to  affect  the  organization  as  much  as  the 


administrative  and  technical  staff.  Within  a  more  technical  organizations,  like  the  one  in  this 


study,  analysts,  scientists,  and  engineers  may  directly  impact  the  products  created  or  the  services 


provided  by  the  organization,  resulting  in  higher  performance  ratings.  Supervisors,  responsible 


for  integrating  the  activities  of  the  previous  jobs  positions,  can  have  the  greatest  impact  in 


affecting  organizational  objectives.  Therefore,  this  relationship  with  higher  performance  ratings 


may  not  indicate  favoritism  for  a  particular  job  position,  but  may  be  caused  by  the  natural 


influence  that  job  category  has  over  the  organization. 


Figure  1 1  (right)  provides  share  split  %  across  job  descriptions.  As  expected,  students 


(YP),  have  near-zero  percentages.  NSPS  is  not  meant  to  be  a  mechanism  for  students  to  gain 


salary  increases.  In  general,  all  of  a  student’s  payout  should  go  into  a  one-time  bonus.  No  other 


relationship  between  job  positions  and  share  split  %  can  be  identified  at  this  time.  It  is 


recommended  that  this  continue  to  be  monitored  in  the  future  to  observe  possible  future  trends. 
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No  Findings 

Several  areas  of  analysis  did  not  find  any  correlations.  Additional  data  was  requested  for 
the  larger  locations  that  had  multiple  pay  pools.  This  data  found  that  pay  pools  at  the  same 
location  were  internally  consistent;  they  shared  properties  with  the  average  across  their  location. 
For  this  reason,  in  the  figures  above,  locations  with  multiple  pay  pools  were  combined  and 
displayed  as  a  unified  average  for  that  location. 

No  unexpected  relationships  were  found  between  the  different  pay  bands  and  the 
resulting  ratings  or  share  percentages.  Pay  band  1  employees,  including  interns  and  students,  did 
show  a  significantly  lower  share  split  %.  This  result  was  expected,  since  NSPS  is  not  meant  to 
provide  salary  increase  for  those  employees.  Pay  band  1  employees  typically  had  all  their  payout 
received  as  a  one-time  bonus. 

Some  relationships  between  share  percentage  splits  and  perfonnance  ratings  were  noted 
in  2009.  Some  pay  pools,  in  2009  gave  higher-rated  employees  a  higher  share  split  %  toward 
salary.  However,  this  practice  was  not  found  in  2010  data.  Many  other  areas  of  analysis  were 
attempted,  but  no  significant  findings  were  observed. 
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CHAPTER  5 
IMPLEMENTATION 


Considering  Herzberg’s  two  factor  theory,  most  pay-for-perfonnance  systems  are  most 
likely  to  affect  employee  motivation  in  two  ways.  Most  important,  employees  wish  to  have  the 
highest  salary  possible;  therefore,  they  will  desire  the  largest  salary  increases  as  result  of  their 
performance.  Categorized  primarily  as  a  dissatisfier,  having  an  equitable  system  to  distribute 
salary  raises  can  avoid  significant  drop  in  employee  morale  from  feelings  of  an  unfair 
performance  appraisal.  Using  the  performance  rating  to  recognize  an  employee’s  contribution  to 
the  command  is  only  a  secondary  effect.  Employees  having  a  high  rating  in  a  system  that  truly 
differentiates  performance  can  be  a  contributing  factor  in  motivating  them  to  work  harder. 
However,  performance  systems  alone  are  more  likely  to  have  a  negative  effect  on  employees  and 
therefore  must  be  used  as  a  part  of  a  larger  coaching  and  leadership  effort  to  motivate  the 
workforce. 

Leadership  is  the  Key  to  Employee  Motivation,  Not  the  Performance  System 

Referring  again  to  Herzberg’s  theory,  some  of  the  greatest  motivators  for  employee  come 
from  the  work  itself,  the  level  of  responsibility  the  employee  believes  he/she  has,  and  his/her 
opportunity  to  achieve.  All  of  these  factors  are  influenced  on  a  daily  bases  by  the  supervisors 
and  organization,  and  are  not  influenced  by  the  performance  management  system.  Other 
motivating  factors  include  recognition  of  achievement,  possibility  of  advancement,  and 
opportunities  for  growth.  Often,  these  motivational  elements  are  overlooked  by  a  supervisor 
rushing  to  complete  numerous  other  organizational  tasks  or  to  complete  documentation  of  the 
appraisal  itself.  However,  the  greatest  opportunity  for  a  perfonnance  management  system  to 
positively  impact  the  workforce  is  if  employee  recognition,  advancement,  and  development  be 
incorporated  by  supervisors.  This  leadership  challenge  of  focusing  on  an  employee’s 
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achievements  and  his/her  future  should  be  addressed  not  only  during  annual  appraisals  but 
routinely  throughout  the  year.  The  pay-for-performance  system  and  the  resulting  annual 
appraisal  should  merely  be  a  “punctuation  point”  emphasizing  the  results  of  perfonnance  to  an 
employee  who  already  should  know  where  he/she  stands.  It  is  during  the  perfonnance  review 
with  the  employee  that  the  supervisor  needs  to  map  out  the  significant  employee  inputs  that  were 
used  to  determine  the  outcomes  of  salary  increase  and  rating.  That  discussion  is  vital  for  the 
employee  to  understand  that  his/her  efforts  and  resulting  performance  review  is  equitable.  The 
salary  increase  and  rating  should  only  be  considered  a  tool  used  for  that  “punctuation.”  The 
AMP-Rev  should  be  considered  a  tool  to  gain  insight  on  those  perfonnance  results. 

Goal  and  Advantage  of  Implementation  Plan 

Pay-for-perfonnance  systems  can  have  a  huge  motivational  effect,  primarily  negative,  on 
workforce  morale  if  executed  improperly.  This  plan  is  proposed  to  provide  a  structural  method, 
using  analysis  metrics,  to  help  ensure  the  performance  ratings  and  financial  rewards  of  the 
performance  management  system  are  distributed  equitably.  By  developing  a  tool,  such  as  AMP- 
Rev,  to  calculate  simple  statistical  analysis  of  performance  review  results,  these  metrics  can  be 
provided  to  senior  leadership  with  minimal  additional  labor.  The  annual  analysis  should  only 
require  a  couple  of  man-days  of  labor  from  human  resources  professionals.  Analysis  during  pay 
pool  reviews  should  happen  in  parallel  of  the  review  process  with  negligible  additional  labor.  By 
perfonning  this  analysis  during  the  pay  pool  review,  it  is  possible  that  a  small  savings  to  the 
organization  will  occur  by  avoiding  inadvertent  extreme  payouts  to  high-performing  employees. 

The  greatest  benefit  of  this  plan  is  in  providing  objective  metrics  that  will  indicate  a  high 
level  of  consistency  of  the  review  system  across  the  organization.  These  metrics  can  then  be 
shared  to  the  workforce  or  other  agencies  providing  a  much  higher  level  of  transparency  of  the 
pay-for-perfonnance  system.  This  level  of  transparency,  exhibiting  highly  consistent  metrics, 
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will  lead  to  greater  trust  and  overall  satisfaction  by  employees,  senior  leaders,  and  any  auditors 
of  the  employed  pay-for-performance  system.  When  an  area  of  improvement  has  been 
identified,  it  will  be  done  early  and  will  illustrate  the  organization’s  proactive  commitment  to 
improving  the  already  consistent  perfonnance  management  system. 

This  implementation  plan  addresses  two  of  the  three  GAO  recommendations  (Ferrell, 
2009,  pg.  50)  to  monitor  the  implementation  of  NSPS  at  the  lower-level  commands  and  ensure 
the  system  is  fair  and  equitable.  Without  an  implementation  plan  such  as  this  one,  details  of  the 
NSPS  implementation  is  likely  to  remain  shrouded  from  employees  and  the  GAO.  This  will 
continue  to  increase  the  level  of  distrust  with  the  system.  However,  with  the  very  small 
investment  leveraging  current  employees’  time  to  perform  this  analysis  and  provide  any 
corrective  guidance,  the  level  of  trust  of  NSPS  will  increase  leading  to  the  pay-for-performance 
as  a  positive  motivational  force  for  employees. 

Employee  Inputs  Considering  Current  Job  Rank  and  Pay  Range 

Equity  theory  states  the  importance  of  matching  employee  inputs  (level  of  effort)  to  their 
outcomes  (perfonnance  rating  and  salary  increase).  Since  the  pay-for-performance  system  also 
matches  input  to  outcome;  both  of  those  pieces  of  information  are  needed  by  AMP-Rev  to 
effective  analyze  performance  results.  Due  to  personal  infonnation  disclosure  limitations, 
infonnation  on  individual  employee  salary  was  released  for  this  study.  However,  this  vital 
infonnation  is  needed  to  do  a  proper  analysis  of  performance  evaluation.  To  understand  the  level 
of  input  an  employee  is  expected  to  perfonn,  their  cunent  job  rank  and  salary  infonnation  is 
required.  Under  the  GS  system,  both  salary  and  rank  was  defined  for  every  employee.  A 
supervisor  may  make  the  salary  of  a  GS  grade  13  step  4,  and  his/her  subordinate  would  have  a 
similar  salary  as  a  GS  grade  12  step  10  (Office  of  Personnel  Management,  2010).  Although 
these  two  employees  make  similar  salary,  different  expectations  in  performance  would  be 
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applied.  Under  NSPS,  the  pay  schedule  and  grade  provides  less  differentiation  between 
employees;  in  this  case  the  supervisor  might  be  a  YC-02  while  his/her  subordinate  a  YA-02. 

The  bands  in  NSPS  were  designed  to  compress  the  highly  categorized  GS  system  with  10 
steps  inside  every  one  of  the  15  grades,  totaling  150  different  ranks.  NSPS  uses  only  3  or  4 
bands,  allowing  for  greater  flexibility  in  assigning  pay.  During  a  pay  pool  review,  it  can  be 
advantageous  to  have  a  greater  degree  of  differentiation  when  determining  an  employee’s 
expected  level  of  input.  Since  most  organizations  within  DoD  have  a  long  history  working  under 
the  GS  system,  it  is  recommended  that  for  each  employee  an  equivalent  GS  grade  and  step  be 
provided  along  with  the  pay-for-perfonnance  job  designation  within  the  AMP-Rev  tool. 

In  GS,  the  range  of  salary  of  a  particular  grade  increases  in  discrete  steps  from  step  1  to 
step  10.  Using  pay  banding,  the  ranges  are  bounded  by  NSPS  rules  and  can  be  additionally 
limited  by  an  organization’s  business  rules.  For  instance  a  salary  range  for  an  entire  pay  band 
may  range  from  $50,000-$75,000.  Yet  an  organization  may  further  designate  control  points  such 
that  a  junior  technician  may  receive  between  $50,000-$60,000,  a  senior  technician  from  $55,000- 
$70,000,  and  a  master  technician  from  $65,000-$75,000.  These  additional  control  points  should 
also  be  listed  within  the  AMP-Rev  tool.  That  way,  a  junior  technician  making  $60,000  and  at  the 
top  of  their  pay  range  will  be  evaluated  differently  than  a  master  technician  at  the  bottom  of  their 
range  with  $65,000.  Having  this  pay  range  infonnation  will  also  prove  vital  in  determining  an 
appropriate  share  split  percentage  once  a  payout  was  calculated.  It  would  be  expected  that,  in 
general,  employees  near  the  top  of  their  established  pay  range  are  likely  to  receive  smaller 
percentages  in  salary  and  more  in  bonuses  than  those  much  lower  in  the  pay  range. 

Employee  Overtime  as  an  Input 

There  can  be  a  debate  on  whether  an  employee  who  works  overtime  has  already  been 
compensated  for  his/her  perfonnance.  As  an  example,  an  employee  who  consistently  works  10 
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hours  of  overtime  every  week  would  be  expected  that  he/she  would  achieve  25  percent  more 
results  through  his/her  50  hours  a  week  than  the  typical  employee  only  working  40  hours.  Since 
he/she  has  already  been  compensated  with  overtime  pay,  that  additional  25  percent  achievement 
may  not  indicate  his/her  performance  exceeded  expectations.  Unless  the  amount  of  overtime  that 
the  employee  earned  over  the  performance  cycle  is  provided  during  the  review,  it  cannot  be 
taken  into  account  as  both  an  employee  input  and  outcome.  Therefore,  it  is  recommended  that 
the  number  of  overtime  hours  be  included  in  the  AMP-Rev  tool  for  every  employee. 

Employee  Outcomes  Using  Salary  Increase  and  Ratings 

All  of  the  findings  within  this  study  referred  to  the  identified  Herzberg  factors  of  payout 
or  salary  increase  and  performance  rating.  This  study  used  a  prototype  of  the  AMP-Rev  tool  and 
collected  over  4,500  data  points  for  employee  perfonnance  ratings  over  the  previous  three  years. 
However,  due  to  personal  infonnation  disclosure,  there  was  no  information  provided  for 
individual  employee  payouts,  salary  increase,  or  bonus  awards.  Since  this  salary  information  is 
the  biggest  factor-relating  perfonnance  systems  and  employee  motivation.  It  is  vital  that  this 
information  be  provided  for  the  AMP-Rev  tool. 

Implementation  of  the  AMP-Rev  tool  must  therefore  be  limited  to  human  resource 
professionals  within  the  organization;  those  that  already  have  a  need-to-know  access  of 
employee  personal  information.  These  professions  could  then  use  the  AMP-Rev  tool  to  identify 
possible  trends  using  the  larger  employee  data  set.  The  tools  could  investigate  variations  in 
performance  ratings  and  current  salary  levels  (do  higher  salaried  employees  typically  receive 
lower  ratings?),  the  percentage  of  the  workforce  that  are  near  the  top  of  their  pay  range,  and 
perhaps,  most  important,  what  are  the  trends  at  the  employee  level  between  their  perfonnance 
and  resulting  salary  increase.  Any  results  can  then  be  presented,  without  the  need  to  divulge  any 
employee-specific  information.  This  trend  analysis  was  done  for  this  study;  human  resource 
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professionals  calculated  average  payout,  salary  increases,  and  bonuses  for  each  of  the  pay  pools 
and  only  presented  that  data.  Those  calculations  were  enough  to  develop  several  findings  within 
the  study  without  the  need  to  provide  employee  personal  information.  Therefore,  the  best 
implementation  of  the  AMP-Rev  tool  is  for  it  to  be  used  only  by  human  resource  professionals  to 
provide  the  initial  analysis  of  trends  with  all  needed  employee  information;  then  presenting  those 
trends  such  that  employee  information  has  been  removed. 

For  organizations  with  a  long  legacy  using  the  GS  system,  it  is  recommended  that 
performance  salary  increase  be  equated  to  the  GS  step  increases  for  each  employee.  A  senior 
employee  who  previously  expected  to  received  a  step  increase  every  three  years  should  be  happy 
to  discover  any  NSPS  payout  resulting  in  a  salary  increase  equivalent  to  a  step  increase  greater 
than  0.33.  By  equating  NSPS  results  to  GS,  employees  will  leam  that  even  with  lower  ratings 
achieved  in  NSPS,  they  are  likely  to  be  financially  better  off  with  the  pay-for-perfonnance 
system.  This  should  also  make  employee  feel  the  new  system  more  equitable  and  can  lead  to 
greater  satisfaction  with  the  perfonnance  management  system. 

Implementation  Under  Different  Pay-For-Performance  Systems,  GS  or  CCAS 

This  study  has  presented  using  the  AMP-Rev  tool  to  focus  on  the  employee  inputs  and 
outcomes  for  the  NSPS  performance  reviews.  However,  this  tool  can  be  used  with  other  pay-for- 
performance  systems.  What  is  required  is  the  appropriate  employee  inputs,  including  his/her  job 
designation,  current  salary,  salary  pay  range,  and  overtime  earned.  Table  4  provides  an  example 
of  how  these  four  employee  inputs  can  be  related  to  three  different  perfonnance  management 
systems:  NSPS,  GS,  and  CCAS. 
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Table  4:  Employee  Inputs  for  Different  Performance  Management  Systems 


Employee 

Job  Description 

Salary  Pay  Range 

Current  Base 

Overtime 

Inputs 

Salary 

Earned 

NSPS 

Pay  Schedule  and 

Pay  band  range  for  YE-02 

$56,991 

520  hrs  last 

Pay  Band 

Ex.  $33,270  -$59,505 

(90.4  %  near  top 

year 

Ex.  YE-02 

of  the  range) 

(10  hrs/week) 

GS 

Grade  and  Step 

Base  Salary  from  GS  1 1 

$56,991 

Same  as 

Ex.  GS  1 1  Step  5 

Step  1  to  Step  10 

(44.4  %  near  top 

above 

Ex.  $50,287 -$65,371 

of  the  range) 

CCAS 

Career  Path  and 

Broadband  range  for  NH-II 

$56,991 

Same  as 

Broadband  Level 

Ex.  $24,43 1-$65, 371 

(79.5  %  near  top 

above 

Ex.  NH-II 

of  the  range) 

Concluded  from  this  study,  the  most  important  performance  outcomes  include 
perfonnance  rating,  salary  increase,  and  bonus  information.  This  information  can  be  provided 
using  most  performance  management  systems  as  shown  in  Table  5.  GS  is  typically  not 
considered  pay-for-performance;  however,  it  could  be  adapted  using  some  of  the  principles  of 
NSPS,  including  evaluating  full-performance  employees  in  pay  pools.  This  would  allow  for 
employees  to  be  matched  with  their  objectives  as  well  as  differentiated  between  each  other. 
Using  a  pay  pool  panel,  more  objective  perfonnance  ratings  could  be  provided  to  employees 
since  supervisors  would  be  able  to  participate  in  the  review  of  all  the  employees  within  a  pay 
pool  and  not  limited  to  only  reviewing  their  direct  subordinates.  Those  employees  designated  as 
high  performers  could  then  be  eligible  for  a  quality  step  increase  (providing  an  increase  in  job 
rank  and  salary)  and/or  a  one-time  bonus  presented  as  a  performance  award.  By  adding  the  time 
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and  rigor  of  a  pay  panel,  the  GS  system  could  gain  the  benefits  learned  from  NSPS  and  be  used 
as  a  pay-for-perfonnance  system.  The  AMP-Rev  tool  could  then  be  used  to  help  monitor  the 
consistency  of  the  resulting  performance  ratings  and  quality  step  increases  in  the  same  way  as 
this  study  reviewed  NSPS  results. 

CCAS  seems  to  bring  an  even  greater  level  of  flexibility  and  variability  in  both 
developing  perfonnance  ratings  and  assigning  financial  benefits  to  the  employee.  As  such,  a  tool 
like  AMP-Rev  could  prove  even  more  vital  to  ensure  consistency  across  and  organization. 

Under  CCAS,  AMP-Rev  would  require  an  additional  input,  each  employee’s  expected  level  of 
contribution.  This  score,  inherently  part  of  CCAS,  provides  a  single  number  of  an  employee’s 
target  performance.  This  expected  contribution  score  articulates  in  one  number  what  the 
elements  of  job  description  and  salary  attempted  to  define  in  NSPS  as  appropriate  employee 
input.  As  an  employee’s  perfonnance  is  rated,  a  resulting  contribution  score  is  calculated.  If  an 
employee’s  calculated  score  is  greater  than  his/her  expected  level  of  contribution,  than  the 
employee  can  be  considered  as  exceeding  expectations  or  “above  the  rail.”  Conversely,  a  lower 
score  indicates  an  employee  may  not  be  working  at  their  appropriate  level.  This  ‘below  the  rail’ 
employee  may  not  receive  a  bonus  nor  a  salary  increase,  and  his/her  pay  could  be  reduced. 
Therefore,  for  any  given  employee,  the  amount  that  his/her  contribution  score  is  above  or  below 
their  expected  score  could  be  used  by  AMP-Rev  to  numerically  analyze  the  employee’s 
perfonnance. 
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Table  5:  Performance  Outcomes  for  Different  Performance  Management  Systems 


Performance 

Outcomes 

Performance  Rating 

Salary  Increase 

Bonus  Award 

NSPS 

Ratings  range  from  highest 

(5)  to  lowest  (1) 

performance 

Percentage  split  of  the 

perfonnance  payout 

towards  annual  salary 

increase. 

Remaining  percentage 

Split  of  the 

performance  payout 

towards  bonus. 

GS 

Ratings  range  from  highest 

(1)  to  lowest  (5) 

performance 

Eligibility  for  a  Quality 

Step  Increase 

Eligibility  for 

Performance  Award 

CCAS 

Calculate  difference 

between  Expected  Level  of 

Contribution  and  resulting 

Contribution  Score 

Eligibility  for  a 

Contribution  Rating 

Increase 

Eligibility  for 

Contribution  Award 

Higher  performance  in  CCAS  can  result  in  both  a  salary  increase  and  a  one-time  bonus. 
These  incentives,  dictated  by  management,  can  be  part  of  the  AMP-Rev’s  analysis  to  ensure 
consistency  of  CCAS  practices  across  a  large  organization. 

Pre-Decisional  Analysis  During  Pay  Pools 

This  study  has  focused  on  the  use  of  the  AMP-Rev  tool  as  a  post-decisional  analysis 
application  to  review  performance  results  for  trends  in  variability.  However,  this  same  tool  is 
recommended  to  be  used  locally  during  the  pay  pool  reviews.  This  allows  not  only  for  real-time 
analysis  of  how  a  pay  pool  is  implementing  any  changes  to  ensure  consistency,  it  can  directly 
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assist  providing  more  equitable  performance  ratings  and  salary  increases.  This  tool  would  enable 
the  pay  pool  to  prevent  undue  performance  rating  tracking,  by  showing  how  rating  scores  have 
increased  since  the  previous  year.  Using  the  input  parameters,  such  as  the  GS  equivalent  grade 
and  step,  a  pay  pool  may  be  able  to  quickly  detennine  an  expected  level  of  perfonnance  for  each 
employee  as  he/she  reviews  his/her  accomplishments.  When  awarding  the  number  of  shares  and 
split  percentages  to  salary,  the  tool  can  help  ensure  a  proper  and  equitable  salary  increase  is 
provided  to  every  employee.  It  can  immediately  inform  the  pay  pool  if  a  “valued  employee” 
may  be  getting  a  smaller  salary  increase  than  appropriate,  particularly  when  compared  to  the  GS 
system.  Likewise,  some  high  performers  may  be  received  unduly  high  salary  increases  when  a 
more  modest  increase  would  be  more  fitting.  Since  the  AMP-Rev  tool  was  designed  to  analyze 
the  consistency  of  the  pay-for-performance  system,  it  may  be  best  used  to  perfonn  that  task  as 
the  reviews  are  being  made. 

Finally,  since  each  location  has  access  to  the  tool,  each  will  know  exactly  what  the 
greater  organization  will  be  using  for  their  analysis  on  consistency.  If  guidance  was  issued  to 
improve  consistency  in  a  certain  area — for  instance,  how  share  split  percentages  should  be 
administered — the  pay  pool  will  be  able  to  analyze  how  they  are  currently  performing  during  that 
perfonnance  review.  If  improvements  can  be  made  during  the  original  pay  pool  review,  this  will 
save  time  rather  than  being  forced  to  reconvene  the  pay  pool  to  make  adjustments. 

Proposed  Timeline  for  Implementation 

The  expected  cost  for  this  plan  is  the  opportunity  cost  by  reassigning  duties  to  current 
employees.  Human  resource  professionals  are  already  assigned  to  collect  data  on  NSPS 
employees  and  perfonn  the  necessary  payroll  actions.  If  it  would  be  possible  to  train  them  on  the 
AMP-Rev  tool  and  allow  them  time  to  use  that  tool  to  analyze  the  data;  only  a  minimal  amount 
of  additional  labor  cost  would  ensue.  The  larger  cost  would  be  the  overhead  labor  for  software 
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engineers  to  develop  a  more  professional  tool  and  to  upgrade  that  tool  part-time.  It  is  possible 
that  a  computer  scientist  student  could  develop  this  tool  as  his/her  summer  project. 

This  plan  would,  therefore,  cause  minimal  impact  to  the  workload  of  many  individuals. 
Computer  scientists  will  have  the  bulk  of  the  workload  developing  the  tool  and  working  with  the 
users  to  provided  necessary  upgrades.  The  human  resource  personnel  already  do  a  lot  of  work 
with  NSPS;  now  they  would  need  to  spend  an  additional  day  or  two  to  perfonn  some  analysis  of 
the  perfonnance  results.  The  pay  pools  would  be  less  impacted;  it  would  likely  require  one 
human  resource  person  in  the  review  dedicated  to  applying  data  into  the  tool  as  the  performance 
reviews  are  conducted.  Finally,  only  a  small  amount  of  additional  effort  would  be  required  by 
the  senior  leadership  to  review  the  analysis  and  provide  necessary  guidance  back  to  the  pay 
pools.  These  four  activities  of  developing  the  tool,  analyzing  the  data,  conducting  performance 
reviews,  and  issuing  strategic  guidance  are  listed  in  the  timeline  of  activities  for  this 
implementation  plan  in  Table  6.  After  the  initial  implementation  year,  an  annual  cycle  will 
become  routine.  Pay  pools  would  use  the  tool  during  the  interim  reviews  (April)  and  annual 
reviews  (November),  the  higher  organization  then  use  the  tool  to  monitor  overall  trends 
(January),  the  senior  leadership  provide  any  corrective  guidance  (July)  and  throughout  the 
application  continues  to  be  upgraded  by  the  development  team. 
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Table  6:  Implementation  Timeline 


Time 

Activity 

Implementation  Step 

May 

Train  & 

Have  organization’s  human  resource  professionals  begin  to  leam  AMP- 

2010 

Analyze 

Rev  prototype.  Add  in  the  recommended  employee  infonnation  and 

perform  similar  analysis  over  the  last  three  years. 

June 

2010 

Develop 

Create  small  team  (possibly  computer  students)  to  modify  AMP-Rev 

prototype  or  develop  new  system,  using  input  from  human  resource 

professionals. 

July 

Strategic 

Present  updated  analysis,  with  enhanced  employee  data,  to  organization 

2010 

Guidance 

leadership.  They  provide  corrective  guidance  to  the  pay  pools. 

Sept 

Develop  & 

Provide  beta  version  of  upgraded  AMP-Rev  tool  to  pay  pools.  Provide 

2010 

Train 

tutorials  and  instruction  using  last  year’s  data  on  the  features  of  the  tool. 

Nov 

Performance 

Pay  pools  use  AMP-Rev  during  performance  reviews  for  pre-decisional 

2010 

Review 

analysis.  Make  note  of  how  the  tool  helped  and  what  variations  are 

endorsed  and  the  situation  around  them. 

Dec 

2010 

Develop 

Pay  pools  provide  feedback  to  AMP-Rev  developing  team  for  further 

upgrades. 

Jan 

2011 

Analyze 

Higher  organization  begins  cross-sectional  and  time-series  analysis  of 

previous  year’s  performance  ratings  across  all  pay  pools. 

Feb 

Develop  & 

Another  revision  of  AMP-Rev  tool  should  be  worked  by  development 

2011 

Strategic 

Guidance 

team.  Leadership  issues  any  guidance  to  pay  pools  from  trends 

presented  by  AMP-Rev. 

Apr 

Performance 

Pay  pools  that  perform  interim  reviews,  will  use  an  updated  version  of 

2010 

Review 

AMP-Rev  during  review  process  and  address  leadership  guidance. 
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Evaluation  of  AMP-Rev’s  Effectiveness 


AMP-Rev  itself  can  be  used  to  determine  its  own  effectiveness.  The  purpose  of  the  tool 
is  to  identify  trends  so  that  they  can  be  corrected.  Subsequent  perfonnance,  reviews  after 
guidance  has  been  provided,  will  already  be  analyzed  and  changes  noted.  If  the  additional 
guidance  is  provided  by  this  organization  to  create  more  consistent  average  performance  ratings 
and  resist  rating  creep  is  successful,  the  result  for  the  following  year  would  expect  that  average 
performance  ratings  across  pay  pools  would  have  less  variance  and  a  lower  overall  average. 

Since  there  would  be  no  evidence  that,  without  guidance,  the  pay  pools  would  continue  to 
slowly  inflate  performance  ratings  and  maintain  high  variability,  an  experiment  could  be  set  up. 
This  would  consist  of  randomly  assigning  pay  pools,  one  group  receiving  guidance  and  the  other 
group  as  control.  The  results  from  the  two  groups  could  then  be  compared  using  AMP-Rev  to 
see  if  that  guidance  caused  in  any  statistical  change.  Although  an  academically  sound 
experiment,  it  is  not  recommended  to  implement  this  since  the  purpose  is  to  decrease  variability 
and  improve  equitable  results  across  all  pay  pools.  It  is  more  important  to  improve  the 
equitability  of  all  pay  pools  than  to  perform  a  viable  experiment.  Therefore,  it  is  recommended 
to  monitor  yearly  results,  make  adjustments  to  all  pay  pools,  and  then  follow  up  to  see  if 
identified  trends  are  counteracted.  A  best  case  example  would  occur  if  every  year  older 
inconsistent  trends  would  be  alleviated  and  new  trends  identified. 

Another  method  to  judge  the  effectiveness  of  AMP-Rev  would  be  to  apply  the  tool  to  a 
different  organization  that  has  archived  their  perfonnance  review  data.  For  instance,  even  a  GS 
organization  could  have  data  on  employee  job  rank,  current  salaries,  and  perfonnance  ratings, 
step  increases,  and  performance  awards  over  several  years.  The  AMP-Rev  could  perform  cross- 
sectional  and  time-series  analysis  and  make  possible  recommendations  for  improvement.  Then, 
in  subsequent  years,  a  time-series  analysis  could  determine  if  positive  changes  occurred.  Since 
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these  changes  occurred  after  the  intervention  of  guidance  developed  using  the  AMP-Rev  tool,  it 
may  be  surmised  that  this  procedure  was  the  major  cause  promoting  the  positive  change. 

Although  changes  in  perfonnance  ratings  over  time  can  be  used  to  judge  the  success  of 
this  tool  as  part  of  an  overall  perfonnance  management  improvement  initiative,  an  external 
survey  or  metric  would  provide  additional  benefit  in  detennining  success.  A  meeting  with  the 
organization’s  human  resource  professionals  could  be  used  to  gain  infonnal  feedback  whether 
the  AMP-Rev  tool  was  used  during  the  pay  pool  reviews,  what  problems  could  be  improved 
upon,  and  what  features  were  most  used  and  possibly  further  enhanced.  A  survey  could  provide 
both  qualitative  and  quantitative  feedback  on  the  use  of  the  tool.  Finally,  if  any  trends, 
conclusions,  or  reports  were  created  by  the  pay  pools  if  they  could  be  reviewed  by  the  higher 
organization  insights  on  how  AMP-Rev  was  used  and  if  it  was  helpful.  In  the  end,  a  user 
community  could  be  established  to  both  determine  AMP-Rev ’s  effectiveness  and  how  best  to 
make  improvements  to  the  tool  or  the  performance  management  implementation  in  general. 
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CHAPTER  6 
RECOMMENDATIONS 


It  is  recommended  that  a  tool  similar  to  AMP-Rev  be  developed  to  be  used  for  both  pre- 
decisional  analysis  at  the  localized  pay  pools  to  monitor  consistency  during  employee 
performance  reviews  and  for  post-decisional  analysis  at  the  higher  command  to  monitor  long¬ 
term  trends  and  issue  guidance  to  ensure  the  implemented  pay-for-perfonnance  system  is  as 
equitable  across  all  locations. 

It  is  recommended  that  the  AMP-Rev  tool  be  used  by  human  resource  personnel  so  that 
employee  sensitive  information  can  be  included  within  the  initial  analysis.  This  additional  data 
includes  current  salary,  awarded  bonuses,  and  salary  increases  for  each  employee.  To  help 
detennine  the  appropriate  level  of  employee  input,  the  number  of  hours  compensated  for 
overtime  should  be  presented  for  each  employee. 

For  commands  that  have  a  long  legacy  using  the  GS  system,  it  is  recommended  that  every 
employee’s  salary  and  job  description  be  equated  to  the  closest  GS  grade  and  step.  The  resulting 
salary  increase  for  each  employee  should  be  converted  to  the  equivalent  step  increase  for  that 
employee’s  approximate  GS  grade  level.  This  way,  explaining  both  what  is  expected  of  an 
employee  and  how  their  performance  resulted  in  a  financial  payout  will  be  easily  understood  by 
both  the  supervisor  and  employee. 

It  is  recommended  that  the  AMP-Rev  tool  be  released  to  the  local  pay  pools  to  assist 
them  in  the  pay  pool  review  process.  The  above  recommendations  using  this  tool  should  be 
leveraged  during  the  perfonnance  evaluation  process.  This  action  will  facilitate  more  equitable 
and  consistent  performance  results  for  the  local  pay  pool  that  should  also  result  in  less  variability 
for  the  overall  organization. 
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The  AMP-Rev  tool  can  also  be  used  with  other  performance  management  systems  by 
incorporating  similar  infonnation  on  employee  inputs  and  perfonnance  outcomes.  Examples  of 
this  data  was  provided  for  the  GS  and  CCAS  perfonnance  systems.  The  tool  could  be  used  with 
the  GS  system,  but  in  addition  it  is  recommended  that  a  pay  pool  be  used  to  review  GS  full 
perfonnance  personnel  making  the  GS  system  more  like  a  pay-for-perfonnance  system. 
Delimitations 

The  AMP-Rev  tool  is  only  a  mathematical  application  that  is  geared  to  look  for  trends 
and  inconsistencies  from  the  results  of  a  perfonnance  system.  As  such,  all  of  the  findings  are 
only  based  on  mathematical  calculations  that  do  not  take  into  account  the  more  important 
leadership  realities  that  may  explain  such  variations.  For  instance,  Finding  1  shows  large 
variations  in  the  perfonnance  scoring  across  pay  pools  and  a  general  trend  indicating  a  slow  rise 
in  those  ratings  over  time.  This  study  is  only  able  to  speculate  on  the  cause  and  effect  decisions 
that  may  have  lead  to  the  data.  The  purpose  of  the  AMP-Rev  tool  is  merely  identifying 
variations  so  that  further  investigation  can  leam  if  that  variation  need  be  corrected.  If  one 
location  had  overcome  a  unique  challenge  in  that  year,  it  would  be  expected  that  its  ratings 
would  be  higher  compared  to  other  locations.  On  the  other  hand,  it  may  also  be  likely  that  a 
location  may  be  losing  some  of  the  rigor  in  its  perfonnance  review  and  allowing  an  increase  in 
ratings  to  occur.  The  AMP-Rev  tool  cannot  make  a  detennination  why  a  trend  occurred,  nor  can 
it  judge  if  the  trend  is  an  inconsistent  practice  that  should  be  remedied  or  a  validate 
implementation  which  that  actually  improved  how  equitable  the  pay-for-perfonnance  was 
executed. 

Due  to  a  lack  of  time  during  the  analysis,  calculations  could  not  be  performed  to  test  for 
statistical  significance  for  the  findings.  Referring  to  Finding  1,  it  would  be  helpful  to  know  that 
the  perfonnance  rating  increase  for  each  pay  pool  was  statistically  significantly  each  year.  If  no 
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statistical  significance  could  be  determined,  then  Finding  1  should  be  redesignated  as  a  “potential 
trend”  rather  than  a  “potential  area  of  improvement.”  It  is  recommended  this  statistical  analysis 
be  perfonned  before  acting  on  any  of  the  findings  within  this  study. 

By  focusing  on  any  variations,  there  can  be  a  tendency  that  the  final  goal  is  blind 
consistency  across  all  variables.  Within  a  given  year,  it  should  be  expected  that  several 
variations  in  the  system  should  occur.  It  would  not  be  equitable  for  employees  if  pay  pools 
drove  doggedly  for  consistency.  Instead,  special  considerations  should  be  encouraged  but 
documented.  For  instance,  if  a  systemic  problem  was  recognized  by  the  pay  pool  as  a 
responsibility  of  supervisors,  it  should  be  encouraged  that  supervisors  for  that  pay  pool  have 
lower  performance  ratings  for  that  tenn.  Using  the  AMP-Rev  tool  may  help  identify  if  such  a 
trend  was  fonning  during  the  pay  pool  review.  Instead  of  the  pay  pool  compensating  and 
adjusting  numbers  to  appear  more  consistent,  it  is  recommended  that  the  perfonnance  ratings 
stand  and  that  the  trend  be  noted  and  endorsed  by  the  pay  pool.  When  post-decisional  analysis 
by  the  higher  command  is  completed,  this  documentation  would  be  used  to  understand  some  of 
the  rolled  up  variations  analyzed  by  AMP-Rev.  The  goal  is  not  to  achieve  a  consistent 
performance  results,  but  to  be  able  to  understand  and  explain  any  variations  so  that  the 
perfonnance  management  system  will  be  seen  as  equitable  by  all  employees. 

Future  Work 

Much  of  the  recommendations  outline  the  proposed  future  enhancements  using  the  AMP- 
Rev  tool.  Most  important,  is  including  statistical  significance  to  mathematically  endorse  any 
findings  mined  by  a  data  analysts  using  AMP-Rev.  The  additional  statistical  rigor  would  greatly 
validate  a  trend  and  punctuate  the  importance  of  a  finding.  Initially,  it  is  recommended  that  the 
results  from  this  study  be  reexamined  using  statistical  significance.  The  AMP-Rev  tool  should 
be  developed  to  automatically  include  the  standard  statistical  packages  like  the  ability  to 
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calculate  pair  wise  T-tests  and  analysis  of  variance  (ANOVA).  In  the  future,  when  a  data  analyst 
indenti  tics  a  potential  finding,  he/she  will  be  able  to  quickly  use  the  statistical  packages  to 
validate  the  statistical  significance  of  that  finding. 

As  noted,  additional  infonnation  on  employee  inputs  and  outcomes  will  provide  a  much 
richer  cross-sectional  analysis.  Providing  the  closest  equivalent  GS  grade  and  step  for  each 
employee  would  allow  for  comparisons  with  other  government  organizations  that  may  use 
different  perfonnance  management  systems.  Since  salary  increase  percentage  is  the  most 
important  factor  to  the  employee.  This  needs  to  be  included  in  the  cross-sectional  analysis  for  all 
employees.  Numerous  checks  on  who  get  the  largest  raises  could  be  performed  to  ensure  the 
most  equitable  procedure  to  distribute  raises  has  occurred. 

This  tool  should  be  provided  to  all  the  pay  pools  within  the  organizations  so  that  they  can 
benefit  looking  at  trends  as  they  proceed  with  performance  reviews.  An  evaluation  metric 
should  be  developed  to  help  determine  the  usefulness  of  the  AMP-Rev  tool  to  those  pay  pools. 

Is  it  being  used?  If  not,  is  it  because  the  tool  is  too  complicated?  What  are  the  best  features  of 
AMP-Rev  used  by  each  pay  pool?  This  information  could  then  be  used  to  make  user  friendly 
modifications  to  the  tool  and  determine  if  pre-decisional  analysis  of  the  perfonnance  results  is 
helpful. 

Finally,  it  would  be  interesting  to  use  AMP-Rev  within  a  different  organization  and 
perhaps  one  with  a  different  perfonnance  management  system.  An  investigation  could  then  be 
executed  on  how  the  two  organizations  approach  performance  management  and  how  different 
pay-for-perfonnance  systems  compare. 
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LIST  OF  ACRONYMS 


AMP-Rev 

Annual  Monitoring  Process  to  Review 

ANOVA 

Analysis  of  Variance 

CCAS 

Contribution-Based  Compensation  and  Appraisal  System 

DoD 

Department  of  Defense 

GAO 

Government  Accountability  Office 

GE 

General  Electric 

GS 

General  Schedule 

HR 

Human  Resource 

NSPS 

National  Security  Personnel  System 

PB 

Pay  Band 

PP 

Pay  Pool 

PS 

Pay  Schedule 

RT 

Rating 

SH 

Shares  Awarded 

S% 

Share  Split  Percentage 

TAPES 

Total  Anny  Performance  Evaluation  System 

WG 

Work  Group 

YR 

Year 
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